niceeval 0.13.1 → 0.13.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (548) hide show
  1. package/INDEX.md +25 -2
  2. package/README.md +9 -10
  3. package/README.zh.md +9 -10
  4. package/dist/agents/ai-sdk.cjs.map +1 -1
  5. package/dist/agents/ai-sdk.d.cts +1 -1
  6. package/dist/agents/ai-sdk.d.mts +1 -1
  7. package/dist/agents/ai-sdk.d.ts +1 -1
  8. package/dist/agents/bub-install-spec.cjs +19 -5
  9. package/dist/agents/bub-install-spec.cjs.map +1 -1
  10. package/dist/agents/bub-install-spec.d.cts +9 -2
  11. package/dist/agents/bub-install-spec.d.mts +9 -2
  12. package/dist/agents/bub-install-spec.d.ts +9 -2
  13. package/dist/agents/bub-install-spec.js +14 -12
  14. package/dist/agents/bub.cjs +10 -6
  15. package/dist/agents/bub.cjs.map +1 -1
  16. package/dist/agents/bub.d.cts +2 -2
  17. package/dist/agents/bub.d.mts +2 -2
  18. package/dist/agents/bub.d.ts +2 -2
  19. package/dist/agents/claude-code.cjs.map +1 -1
  20. package/dist/agents/claude-code.d.cts +2 -2
  21. package/dist/agents/claude-code.d.mts +2 -2
  22. package/dist/agents/claude-code.d.ts +2 -2
  23. package/dist/agents/codex.cjs +4 -4
  24. package/dist/agents/codex.cjs.map +1 -1
  25. package/dist/agents/codex.d.cts +3 -3
  26. package/dist/agents/codex.d.mts +3 -3
  27. package/dist/agents/codex.d.ts +3 -3
  28. package/dist/agents/coding-cli-versions.cjs +10 -1
  29. package/dist/agents/coding-cli-versions.cjs.map +1 -1
  30. package/dist/agents/coding-cli-versions.d.cts +8 -3
  31. package/dist/agents/coding-cli-versions.d.mts +8 -3
  32. package/dist/agents/coding-cli-versions.d.ts +8 -3
  33. package/dist/agents/coding-cli-versions.js +18 -12
  34. package/dist/agents/deepseek-harness.cjs +151 -0
  35. package/dist/agents/deepseek-harness.cjs.map +1 -0
  36. package/dist/agents/deepseek-harness.d.cts +14 -0
  37. package/dist/agents/deepseek-harness.d.mts +14 -0
  38. package/dist/agents/deepseek-harness.d.ts +14 -0
  39. package/dist/agents/deepseek-harness.js +7 -0
  40. package/dist/agents/deepseek-harness.js.map +1 -0
  41. package/dist/agents/exact-npm-plugins.cjs +28 -0
  42. package/dist/agents/exact-npm-plugins.cjs.map +1 -0
  43. package/dist/agents/exact-npm-plugins.d.cts +10 -0
  44. package/dist/agents/exact-npm-plugins.d.mts +10 -0
  45. package/dist/agents/exact-npm-plugins.d.ts +10 -0
  46. package/dist/agents/exact-npm-plugins.js +8 -0
  47. package/dist/agents/exact-npm-plugins.js.map +1 -0
  48. package/dist/agents/index.cjs +5 -1
  49. package/dist/agents/index.cjs.map +1 -1
  50. package/dist/agents/index.d.cts +4 -0
  51. package/dist/agents/index.d.mts +4 -0
  52. package/dist/agents/index.d.ts +4 -0
  53. package/dist/agents/index.js +54 -50
  54. package/dist/agents/index.mjs +54 -50
  55. package/dist/agents/omp.cjs +170 -0
  56. package/dist/agents/omp.cjs.map +1 -0
  57. package/dist/agents/omp.d.cts +14 -0
  58. package/dist/agents/omp.d.mts +14 -0
  59. package/dist/agents/omp.d.ts +14 -0
  60. package/dist/agents/omp.js +7 -0
  61. package/dist/agents/omp.js.map +1 -0
  62. package/dist/agents/openclaw.cjs +51 -2
  63. package/dist/agents/openclaw.cjs.map +1 -1
  64. package/dist/agents/openclaw.d.cts +2 -0
  65. package/dist/agents/openclaw.d.mts +2 -0
  66. package/dist/agents/openclaw.d.ts +2 -0
  67. package/dist/agents/types.cjs.map +1 -1
  68. package/dist/agents/types.d.cts +4 -4
  69. package/dist/agents/types.d.mts +4 -4
  70. package/dist/agents/types.d.ts +4 -4
  71. package/dist/agents/ui-message-stream.cjs +24 -7
  72. package/dist/agents/ui-message-stream.cjs.map +1 -1
  73. package/dist/analysis/api.cjs +28 -2
  74. package/dist/analysis/api.cjs.map +1 -1
  75. package/dist/analysis/api.d.cts +22 -1
  76. package/dist/analysis/api.d.mts +22 -1
  77. package/dist/analysis/api.d.ts +22 -1
  78. package/dist/analysis/api.js +8 -6
  79. package/dist/analysis/bindings.cjs +11 -18
  80. package/dist/analysis/bindings.cjs.map +1 -1
  81. package/dist/analysis/bindings.d.cts +177 -49
  82. package/dist/analysis/bindings.d.mts +177 -49
  83. package/dist/analysis/bindings.d.ts +177 -49
  84. package/dist/analysis/contracts.cjs +1 -0
  85. package/dist/analysis/contracts.cjs.map +1 -1
  86. package/dist/analysis/contracts.d.cts +43 -3
  87. package/dist/analysis/contracts.d.mts +43 -3
  88. package/dist/analysis/contracts.d.ts +43 -3
  89. package/dist/analysis/cost-projection.cjs +2 -1
  90. package/dist/analysis/cost-projection.cjs.map +1 -1
  91. package/dist/analysis/cost-projection.d.cts +1 -1
  92. package/dist/analysis/cost-projection.d.mts +1 -1
  93. package/dist/analysis/cost-projection.d.ts +1 -1
  94. package/dist/analysis/cost.cjs +13 -3
  95. package/dist/analysis/cost.cjs.map +1 -1
  96. package/dist/analysis/cost.d.cts +10 -3
  97. package/dist/analysis/cost.d.mts +10 -3
  98. package/dist/analysis/cost.d.ts +10 -3
  99. package/dist/analysis/experiment-groups.cjs +96 -0
  100. package/dist/analysis/experiment-groups.cjs.map +1 -0
  101. package/dist/analysis/experiment-groups.d.cts +9 -0
  102. package/dist/analysis/experiment-groups.d.mts +9 -0
  103. package/dist/analysis/experiment-groups.d.ts +9 -0
  104. package/dist/analysis/experiment-groups.js +10 -0
  105. package/dist/analysis/experiment-groups.js.map +1 -0
  106. package/dist/analysis/index.cjs +4 -1
  107. package/dist/analysis/index.cjs.map +1 -1
  108. package/dist/analysis/index.d.cts +4 -2
  109. package/dist/analysis/index.d.mts +4 -2
  110. package/dist/analysis/index.d.ts +4 -2
  111. package/dist/analysis/index.js +40 -36
  112. package/dist/analysis/index.mjs +40 -36
  113. package/dist/assertions/limits.cjs +4 -0
  114. package/dist/assertions/limits.cjs.map +1 -1
  115. package/dist/assertions/limits.d.cts +4 -0
  116. package/dist/assertions/limits.d.mts +4 -0
  117. package/dist/assertions/limits.d.ts +4 -0
  118. package/dist/assertions/match.cjs +63 -25
  119. package/dist/assertions/match.cjs.map +1 -1
  120. package/dist/assertions/match.d.cts +7 -0
  121. package/dist/assertions/match.d.mts +7 -0
  122. package/dist/assertions/match.d.ts +7 -0
  123. package/dist/assertions/record/attachment.cjs +32 -4
  124. package/dist/assertions/record/attachment.cjs.map +1 -1
  125. package/dist/assertions/record/diff.cjs +10 -4
  126. package/dist/assertions/record/diff.cjs.map +1 -1
  127. package/dist/assertions/record/producer.cjs +101 -4
  128. package/dist/assertions/record/producer.cjs.map +1 -1
  129. package/dist/assertions/record/producer.d.cts +4 -1
  130. package/dist/assertions/record/producer.d.mts +4 -1
  131. package/dist/assertions/record/producer.d.ts +4 -1
  132. package/dist/assertions/runtime.cjs +45 -5
  133. package/dist/assertions/runtime.cjs.map +1 -1
  134. package/dist/cli/record.cjs +113 -8
  135. package/dist/cli/record.cjs.map +1 -1
  136. package/dist/cli/record.d.cts +20 -1
  137. package/dist/cli/record.d.mts +20 -1
  138. package/dist/cli/record.d.ts +20 -1
  139. package/dist/cli/record.js +4 -2
  140. package/dist/cli.cjs +386 -31
  141. package/dist/cli.cjs.map +1 -1
  142. package/dist/cli.d.cts +10 -2
  143. package/dist/cli.d.mts +10 -2
  144. package/dist/cli.d.ts +10 -2
  145. package/dist/context/send-retry.cjs +4 -4
  146. package/dist/context/send-retry.cjs.map +1 -1
  147. package/dist/coordination/host/runtime.cjs +8 -30
  148. package/dist/coordination/host/runtime.cjs.map +1 -1
  149. package/dist/coordination/host/runtime.d.cts +3 -3
  150. package/dist/coordination/host/runtime.d.mts +3 -3
  151. package/dist/coordination/host/runtime.d.ts +3 -3
  152. package/dist/coordination/host/types.cjs.map +1 -1
  153. package/dist/coordination/host/types.d.cts +0 -5
  154. package/dist/coordination/host/types.d.mts +0 -5
  155. package/dist/coordination/host/types.d.ts +0 -5
  156. package/dist/coordination/platform/node.cjs +90 -9
  157. package/dist/coordination/platform/node.cjs.map +1 -1
  158. package/dist/define.cjs +25 -1
  159. package/dist/define.cjs.map +1 -1
  160. package/dist/experiment/host/index.cjs +1 -0
  161. package/dist/experiment/host/index.cjs.map +1 -1
  162. package/dist/i18n/en.cjs +26 -8
  163. package/dist/i18n/en.cjs.map +1 -1
  164. package/dist/i18n/en.d.cts +19 -6
  165. package/dist/i18n/en.d.mts +19 -6
  166. package/dist/i18n/en.d.ts +19 -6
  167. package/dist/index.cjs.map +1 -1
  168. package/dist/index.d.cts +1 -1
  169. package/dist/index.d.mts +1 -1
  170. package/dist/index.d.ts +1 -1
  171. package/dist/o11y/parsers/hermes.cjs +1 -1
  172. package/dist/o11y/parsers/hermes.cjs.map +1 -1
  173. package/dist/o11y/prices.json +1 -1
  174. package/dist/o11y/record/families.cjs +12 -3
  175. package/dist/o11y/record/families.cjs.map +1 -1
  176. package/dist/o11y/record/families.d.cts +8 -8
  177. package/dist/o11y/record/families.d.mts +8 -8
  178. package/dist/o11y/record/families.d.ts +8 -8
  179. package/dist/o11y/record/family-projectors.d.cts +66 -18
  180. package/dist/o11y/record/family-projectors.d.mts +66 -18
  181. package/dist/o11y/record/family-projectors.d.ts +66 -18
  182. package/dist/o11y/record/family-writers.d.cts +11 -3
  183. package/dist/o11y/record/family-writers.d.mts +11 -3
  184. package/dist/o11y/record/family-writers.d.ts +11 -3
  185. package/dist/o11y/record/runner-producer.cjs +14 -4
  186. package/dist/o11y/record/runner-producer.cjs.map +1 -1
  187. package/dist/record/codec/core.d.cts +2 -2
  188. package/dist/record/codec/core.d.mts +2 -2
  189. package/dist/record/codec/core.d.ts +2 -2
  190. package/dist/record/family/catalog.d.cts +44 -12
  191. package/dist/record/family/catalog.d.mts +44 -12
  192. package/dist/record/family/catalog.d.ts +44 -12
  193. package/dist/record/family/index.js +128 -124
  194. package/dist/record/family/observability-v1.cjs +35 -0
  195. package/dist/record/family/observability-v1.cjs.map +1 -0
  196. package/dist/record/family/observability-v1.d.cts +3 -0
  197. package/dist/record/family/observability-v1.d.mts +3 -0
  198. package/dist/record/family/observability-v1.d.ts +3 -0
  199. package/dist/record/family/observability-v1.js +6 -0
  200. package/dist/record/family/observability-v1.js.map +1 -0
  201. package/dist/record/family/observability.cjs +58 -9
  202. package/dist/record/family/observability.cjs.map +1 -1
  203. package/dist/record/family/observability.d.cts +1014 -15
  204. package/dist/record/family/observability.d.mts +1014 -15
  205. package/dist/record/family/observability.d.ts +1014 -15
  206. package/dist/record/family/observability.js +22 -18
  207. package/dist/record/host/runtime.cjs +600 -31
  208. package/dist/record/host/runtime.cjs.map +1 -1
  209. package/dist/record/host/types.cjs.map +1 -1
  210. package/dist/record/host/types.d.cts +33 -9
  211. package/dist/record/host/types.d.mts +33 -9
  212. package/dist/record/host/types.d.ts +33 -9
  213. package/dist/record/maintenance/runtime.cjs +1 -2
  214. package/dist/record/maintenance/runtime.cjs.map +1 -1
  215. package/dist/record/model/core.cjs.map +1 -1
  216. package/dist/record/model/core.d.cts +1 -1
  217. package/dist/record/model/core.d.mts +1 -1
  218. package/dist/record/model/core.d.ts +1 -1
  219. package/dist/record/model/definition.d.cts +6 -6
  220. package/dist/record/model/definition.d.mts +6 -6
  221. package/dist/record/model/definition.d.ts +6 -6
  222. package/dist/record/model/identifiers.cjs +1 -1
  223. package/dist/record/model/identifiers.cjs.map +1 -1
  224. package/dist/record/model/identifiers.d.cts +1 -1
  225. package/dist/record/model/identifiers.d.mts +1 -1
  226. package/dist/record/model/identifiers.d.ts +1 -1
  227. package/dist/record/platform/node.cjs +159 -23
  228. package/dist/record/platform/node.cjs.map +1 -1
  229. package/dist/record/platform/services.cjs.map +1 -1
  230. package/dist/record/platform/services.d.cts +15 -2
  231. package/dist/record/platform/services.d.mts +15 -2
  232. package/dist/record/platform/services.d.ts +15 -2
  233. package/dist/record/reader/errors.cjs +37 -1
  234. package/dist/record/reader/errors.cjs.map +1 -1
  235. package/dist/record/reader/errors.d.cts +49 -1
  236. package/dist/record/reader/errors.d.mts +49 -1
  237. package/dist/record/reader/errors.d.ts +49 -1
  238. package/dist/record/reader/errors.js +22 -12
  239. package/dist/record/reader/format.cjs +76 -17
  240. package/dist/record/reader/format.cjs.map +1 -1
  241. package/dist/record/reader/format.d.cts +11 -4
  242. package/dist/record/reader/format.d.mts +11 -4
  243. package/dist/record/reader/format.d.ts +11 -4
  244. package/dist/record/reader/format.js +2 -0
  245. package/dist/record/reader/index.js +32 -18
  246. package/dist/record/reader/runtime.cjs +64 -0
  247. package/dist/record/reader/runtime.cjs.map +1 -1
  248. package/dist/record/reader/runtime.d.cts +15 -0
  249. package/dist/record/reader/runtime.d.mts +15 -0
  250. package/dist/record/reader/runtime.d.ts +15 -0
  251. package/dist/record/reader/runtime.js +2 -0
  252. package/dist/report/assets/styles.css +53 -29
  253. package/dist/report/built-in/analysis-values.cjs +21 -13
  254. package/dist/report/built-in/analysis-values.cjs.map +1 -1
  255. package/dist/report/built-in/analysis-values.d.cts +11 -22
  256. package/dist/report/built-in/analysis-values.d.mts +11 -22
  257. package/dist/report/built-in/analysis-values.d.ts +11 -22
  258. package/dist/report/built-in/analysis-values.js +4 -6
  259. package/dist/report/built-in/index.cjs +1 -3
  260. package/dist/report/built-in/index.cjs.map +1 -1
  261. package/dist/report/built-in/index.d.cts +1 -1
  262. package/dist/report/built-in/index.d.mts +1 -1
  263. package/dist/report/built-in/index.d.ts +1 -1
  264. package/dist/report/built-in/index.js +4 -8
  265. package/dist/report/built-in/index.mjs +4 -8
  266. package/dist/report/built-in/machine.cjs +0 -2
  267. package/dist/report/built-in/machine.cjs.map +1 -1
  268. package/dist/report/built-in/machine.d.cts +0 -2
  269. package/dist/report/built-in/machine.d.mts +0 -2
  270. package/dist/report/built-in/machine.d.ts +0 -2
  271. package/dist/report/built-in/result-components.cjs +49 -51
  272. package/dist/report/built-in/result-components.cjs.map +1 -1
  273. package/dist/report/built-in/result-components.d.cts +16 -17
  274. package/dist/report/built-in/result-components.d.mts +16 -17
  275. package/dist/report/built-in/result-components.d.ts +16 -17
  276. package/dist/report/built-in/result-components.js +0 -4
  277. package/dist/report/built-in/run-membership-overview.cjs +126 -17
  278. package/dist/report/built-in/run-membership-overview.cjs.map +1 -1
  279. package/dist/report/built-in/run-membership-overview.d.cts +14 -0
  280. package/dist/report/built-in/run-membership-overview.d.mts +14 -0
  281. package/dist/report/built-in/run-membership-overview.d.ts +14 -0
  282. package/dist/report/built-in/run-membership-overview.js +4 -2
  283. package/dist/report/built-in/standard.cjs +80 -45
  284. package/dist/report/built-in/standard.cjs.map +1 -1
  285. package/dist/report/built-in/standard.d.cts +19 -41
  286. package/dist/report/built-in/standard.d.mts +19 -41
  287. package/dist/report/built-in/standard.d.ts +19 -41
  288. package/dist/report/built-in/standard.js +6 -6
  289. package/dist/report/components/attempt-detail/compute.cjs +78 -2
  290. package/dist/report/components/attempt-detail/compute.cjs.map +1 -1
  291. package/dist/report/components/attempt-detail/compute.d.cts +1 -1
  292. package/dist/report/components/attempt-detail/compute.d.mts +1 -1
  293. package/dist/report/components/attempt-detail/compute.d.ts +1 -1
  294. package/dist/report/components/attempt-detail/content.cjs +1 -1
  295. package/dist/report/components/attempt-detail/content.cjs.map +1 -1
  296. package/dist/report/components/entity-lists/compute.cjs +42 -5
  297. package/dist/report/components/entity-lists/compute.cjs.map +1 -1
  298. package/dist/report/components/entity-lists/compute.d.cts +10 -8
  299. package/dist/report/components/entity-lists/compute.d.mts +10 -8
  300. package/dist/report/components/entity-lists/compute.d.ts +10 -8
  301. package/dist/report/components/entity-lists/content.cjs +31 -6
  302. package/dist/report/components/entity-lists/content.cjs.map +1 -1
  303. package/dist/report/components/entity-lists/index.cjs +9 -4
  304. package/dist/report/components/entity-lists/index.cjs.map +1 -1
  305. package/dist/report/components/entity-lists/index.d.cts +4 -7
  306. package/dist/report/components/entity-lists/index.d.mts +4 -7
  307. package/dist/report/components/entity-lists/index.d.ts +4 -7
  308. package/dist/report/components/summaries/compute.cjs +8 -1
  309. package/dist/report/components/summaries/compute.cjs.map +1 -1
  310. package/dist/report/components/summaries/compute.d.cts +9 -1
  311. package/dist/report/components/summaries/compute.d.mts +9 -1
  312. package/dist/report/components/summaries/compute.d.ts +9 -1
  313. package/dist/report/components/summaries/index.cjs +48 -4
  314. package/dist/report/components/summaries/index.cjs.map +1 -1
  315. package/dist/report/components/summaries/index.d.cts +7 -4
  316. package/dist/report/components/summaries/index.d.mts +7 -4
  317. package/dist/report/components/summaries/index.d.ts +7 -4
  318. package/dist/report/definition/cell.cjs +3 -0
  319. package/dist/report/definition/cell.cjs.map +1 -1
  320. package/dist/report/definition/cell.d.cts +2 -0
  321. package/dist/report/definition/cell.d.mts +2 -0
  322. package/dist/report/definition/cell.d.ts +2 -0
  323. package/dist/report/definition/primitives/chart.cjs +8 -5
  324. package/dist/report/definition/primitives/chart.cjs.map +1 -1
  325. package/dist/report/definition/primitives/points-dataset.cjs +1 -0
  326. package/dist/report/definition/primitives/points-dataset.cjs.map +1 -1
  327. package/dist/report/definition/primitives.cjs +15 -2
  328. package/dist/report/definition/primitives.cjs.map +1 -1
  329. package/dist/report/definition/report.cjs +27 -3
  330. package/dist/report/definition/report.cjs.map +1 -1
  331. package/dist/report/definition/report.d.cts +8 -0
  332. package/dist/report/definition/report.d.mts +8 -0
  333. package/dist/report/definition/report.d.ts +8 -0
  334. package/dist/report/execution/machine.cjs +30 -14
  335. package/dist/report/execution/machine.cjs.map +1 -1
  336. package/dist/report/execution/machine.d.cts +20 -10
  337. package/dist/report/execution/machine.d.mts +20 -10
  338. package/dist/report/execution/machine.d.ts +20 -10
  339. package/dist/report/execution/machine.js +6 -6
  340. package/dist/report/host/execute.cjs +25 -16
  341. package/dist/report/host/execute.cjs.map +1 -1
  342. package/dist/report/host/execute.d.cts +4 -0
  343. package/dist/report/host/execute.d.mts +4 -0
  344. package/dist/report/host/execute.d.ts +4 -0
  345. package/dist/report/host/from-record.cjs +22 -5
  346. package/dist/report/host/from-record.cjs.map +1 -1
  347. package/dist/report/host/machine.cjs +53 -11
  348. package/dist/report/host/machine.cjs.map +1 -1
  349. package/dist/report/host/site-runtime.cjs +9 -8
  350. package/dist/report/host/site-runtime.cjs.map +1 -1
  351. package/dist/report/host/site-runtime.d.cts +1 -1
  352. package/dist/report/host/site-runtime.d.mts +1 -1
  353. package/dist/report/host/site-runtime.d.ts +1 -1
  354. package/dist/report/host/static.cjs +56 -9
  355. package/dist/report/host/static.cjs.map +1 -1
  356. package/dist/report/index.cjs.map +1 -1
  357. package/dist/report/index.d.cts +1 -1
  358. package/dist/report/index.d.mts +1 -1
  359. package/dist/report/index.d.ts +1 -1
  360. package/dist/report/model/dataset.cjs +7 -2
  361. package/dist/report/model/dataset.cjs.map +1 -1
  362. package/dist/report/model/locale.cjs +2 -0
  363. package/dist/report/model/locale.cjs.map +1 -1
  364. package/dist/report/model/locale.d.cts +1 -0
  365. package/dist/report/model/locale.d.mts +1 -0
  366. package/dist/report/model/locale.d.ts +1 -0
  367. package/dist/report/model/metrics.cjs +3 -1
  368. package/dist/report/model/metrics.cjs.map +1 -1
  369. package/dist/report/model/pricing.cjs +16 -11
  370. package/dist/report/model/pricing.cjs.map +1 -1
  371. package/dist/report/react/styles.css +53 -29
  372. package/dist/runner/adoption.cjs +1 -0
  373. package/dist/runner/adoption.cjs.map +1 -1
  374. package/dist/runner/attempt.cjs +44 -5
  375. package/dist/runner/attempt.cjs.map +1 -1
  376. package/dist/runner/attempt.d.cts +13 -1
  377. package/dist/runner/attempt.d.mts +13 -1
  378. package/dist/runner/attempt.d.ts +13 -1
  379. package/dist/runner/config-identity.cjs +10 -2
  380. package/dist/runner/config-identity.cjs.map +1 -1
  381. package/dist/runner/config-identity.d.cts +4 -2
  382. package/dist/runner/config-identity.d.mts +4 -2
  383. package/dist/runner/config-identity.d.ts +4 -2
  384. package/dist/runner/feedback/human.cjs +298 -59
  385. package/dist/runner/feedback/human.cjs.map +1 -1
  386. package/dist/runner/feedback/json.cjs +14 -6
  387. package/dist/runner/feedback/json.cjs.map +1 -1
  388. package/dist/runner/feedback/json.d.cts +3 -2
  389. package/dist/runner/feedback/json.d.mts +3 -2
  390. package/dist/runner/feedback/json.d.ts +3 -2
  391. package/dist/runner/feedback/reducer.cjs +2 -0
  392. package/dist/runner/feedback/reducer.cjs.map +1 -1
  393. package/dist/runner/judge-config.cjs +1 -1
  394. package/dist/runner/judge-config.cjs.map +1 -1
  395. package/dist/runner/ledger.cjs +124 -12
  396. package/dist/runner/ledger.cjs.map +1 -1
  397. package/dist/runner/project-current.cjs +13 -5
  398. package/dist/runner/project-current.cjs.map +1 -1
  399. package/dist/runner/run.cjs +564 -256
  400. package/dist/runner/run.cjs.map +1 -1
  401. package/dist/runner/sandbox-pool.cjs +161 -30
  402. package/dist/runner/sandbox-pool.cjs.map +1 -1
  403. package/dist/runner/sandbox-pool.d.cts +34 -2
  404. package/dist/runner/sandbox-pool.d.mts +34 -2
  405. package/dist/runner/sandbox-pool.d.ts +34 -2
  406. package/dist/runner/shared-state-lease.cjs +803 -0
  407. package/dist/runner/shared-state-lease.cjs.map +1 -0
  408. package/dist/runner/shared-state-lease.d.cts +121 -0
  409. package/dist/runner/shared-state-lease.d.mts +121 -0
  410. package/dist/runner/shared-state-lease.d.ts +121 -0
  411. package/dist/runner/shared-state-lease.js +24 -0
  412. package/dist/runner/shared-state-lease.js.map +1 -0
  413. package/dist/runner/teardown-registry.cjs +97 -34
  414. package/dist/runner/teardown-registry.cjs.map +1 -1
  415. package/dist/runner/teardown-registry.d.cts +27 -1
  416. package/dist/runner/teardown-registry.d.mts +27 -1
  417. package/dist/runner/teardown-registry.d.ts +27 -1
  418. package/dist/runner/teardown-registry.js +20 -16
  419. package/dist/runner/types.cjs.map +1 -1
  420. package/dist/runner/types.d.cts +31 -5
  421. package/dist/runner/types.d.mts +31 -5
  422. package/dist/runner/types.d.ts +31 -5
  423. package/dist/sample/analysis.cjs +249 -459
  424. package/dist/sample/analysis.cjs.map +1 -1
  425. package/dist/sample/capability.cjs +138 -7
  426. package/dist/sample/capability.cjs.map +1 -1
  427. package/dist/sample/capability.d.cts +3 -1
  428. package/dist/sample/capability.d.mts +3 -1
  429. package/dist/sample/capability.d.ts +3 -1
  430. package/dist/sample/capability.js +8 -6
  431. package/dist/sandbox/docker-agent-image.cjs +5 -1
  432. package/dist/sandbox/docker-agent-image.cjs.map +1 -1
  433. package/dist/sandbox/docker-agent-image.d.cts +2 -0
  434. package/dist/sandbox/docker-agent-image.d.mts +2 -0
  435. package/dist/sandbox/docker-agent-image.d.ts +2 -0
  436. package/dist/sandbox/docker-agent-image.js +14 -10
  437. package/dist/sandbox/docker-profile/schema.cjs +283 -534
  438. package/dist/sandbox/docker-profile/schema.cjs.map +1 -1
  439. package/dist/sandbox/docker-profile/schema.d.cts +10 -27
  440. package/dist/sandbox/docker-profile/schema.d.mts +10 -27
  441. package/dist/sandbox/docker-profile/schema.d.ts +10 -27
  442. package/dist/sandbox/e2b-agent-template.cjs +5 -2
  443. package/dist/sandbox/e2b-agent-template.cjs.map +1 -1
  444. package/dist/sandbox/index.cjs +4 -2
  445. package/dist/sandbox/index.cjs.map +1 -1
  446. package/dist/sandbox/index.d.cts +1 -1
  447. package/dist/sandbox/index.d.mts +1 -1
  448. package/dist/sandbox/index.d.ts +1 -1
  449. package/dist/sandbox/index.js +120 -116
  450. package/dist/sandbox/index.mjs +120 -116
  451. package/dist/sandbox/keep-registry.cjs +28 -66
  452. package/dist/sandbox/keep-registry.cjs.map +1 -1
  453. package/dist/sandbox/types.cjs.map +1 -1
  454. package/dist/sandbox/types.d.cts +1 -1
  455. package/dist/sandbox/types.d.mts +1 -1
  456. package/dist/sandbox/types.d.ts +1 -1
  457. package/dist/shared/aggregate.cjs +16 -6
  458. package/dist/shared/aggregate.cjs.map +1 -1
  459. package/dist/shared/aggregate.d.cts +12 -4
  460. package/dist/shared/aggregate.d.mts +12 -4
  461. package/dist/shared/aggregate.d.ts +12 -4
  462. package/dist/shared/turn-label.cjs +20 -2
  463. package/dist/shared/turn-label.cjs.map +1 -1
  464. package/dist/shared/turn-label.d.cts +2 -0
  465. package/dist/shared/turn-label.d.mts +2 -0
  466. package/dist/shared/turn-label.d.ts +2 -0
  467. package/dist/shared/turn-label.js +4 -2
  468. package/dist/vendor/remark/remark-gfm.cjs +7 -7
  469. package/dist/vendor/remark/remark-gfm.cjs.map +1 -1
  470. package/dist/vendor/remark/remark-parse.cjs +12 -12
  471. package/dist/vendor/remark/remark-parse.cjs.map +1 -1
  472. package/dist/vendor/remark/remark.cjs +13 -13
  473. package/dist/vendor/remark/remark.cjs.map +1 -1
  474. package/dist/view/server.cjs +6 -0
  475. package/dist/view/server.cjs.map +1 -1
  476. package/docs-site/images/hitl-handshake-zh.svg +3 -3
  477. package/docs-site/zh/examples/ai-agent-application.mdx +1 -1
  478. package/docs-site/zh/examples/coding-agent-extensions.mdx +1 -1
  479. package/docs-site/zh/examples/integrations/ai-sdk-v7.mdx +4 -3
  480. package/docs-site/zh/examples/integrations/claude-sdk.mdx +3 -3
  481. package/docs-site/zh/examples/integrations/codex-sdk.mdx +5 -5
  482. package/docs-site/zh/examples/integrations/langgraph.mdx +4 -4
  483. package/docs-site/zh/examples/integrations/pi-sdk.mdx +5 -5
  484. package/docs-site/zh/explanation/adapter.mdx +5 -5
  485. package/docs-site/zh/explanation/assert.mdx +7 -7
  486. package/docs-site/zh/explanation/drive.mdx +4 -2
  487. package/docs-site/zh/explanation/evals.mdx +10 -8
  488. package/docs-site/zh/explanation/experiment.mdx +2 -2
  489. package/docs-site/zh/explanation/hitl.mdx +9 -9
  490. package/docs-site/zh/explanation/judge.mdx +5 -5
  491. package/docs-site/zh/explanation/overview.mdx +3 -3
  492. package/docs-site/zh/explanation/runner.mdx +1 -1
  493. package/docs-site/zh/index.mdx +3 -3
  494. package/docs-site/zh/reference/builtin-agents.mdx +13 -13
  495. package/docs-site/zh/reference/capabilities.mdx +1 -1
  496. package/docs-site/zh/reference/cli.mdx +18 -12
  497. package/docs-site/zh/reference/define-agent.mdx +130 -5
  498. package/docs-site/zh/reference/define-config.mdx +2 -2
  499. package/docs-site/zh/reference/define-eval.mdx +4 -4
  500. package/docs-site/zh/reference/events.mdx +11 -11
  501. package/docs-site/zh/reference/official-adapters.mdx +6 -6
  502. package/docs-site/zh/reference/results-data.mdx +6 -0
  503. package/docs-site/zh/troubleshooting/debugging.mdx +1 -1
  504. package/docs-site/zh/troubleshooting/recover-after-kill.mdx +37 -5
  505. package/docs-site/zh/tutorials/accept-results.mdx +72 -0
  506. package/docs-site/zh/tutorials/agent-onboarding.mdx +6 -6
  507. package/docs-site/zh/tutorials/authoring.mdx +9 -13
  508. package/docs-site/zh/tutorials/ci-integration.mdx +6 -6
  509. package/docs-site/zh/tutorials/compare-runs.mdx +61 -0
  510. package/docs-site/zh/tutorials/concurrency.mdx +13 -12
  511. package/docs-site/zh/tutorials/configuration.mdx +4 -6
  512. package/docs-site/zh/tutorials/connect-your-agent.mdx +1 -1
  513. package/docs-site/zh/tutorials/control-run-cost.mdx +58 -0
  514. package/docs-site/zh/tutorials/criteria-files.mdx +2 -2
  515. package/docs-site/zh/tutorials/custom-reports.mdx +2 -2
  516. package/docs-site/zh/tutorials/debug-lifecycle-plan.mdx +46 -0
  517. package/docs-site/zh/tutorials/docker-in-docker.mdx +52 -43
  518. package/docs-site/zh/tutorials/eval-groups.mdx +6 -6
  519. package/docs-site/zh/tutorials/evaluation-kinds.mdx +3 -3
  520. package/docs-site/zh/tutorials/experiments.mdx +2 -3
  521. package/docs-site/zh/tutorials/handle-execution-failures.mdx +18 -10
  522. package/docs-site/zh/tutorials/install-custom-sandbox-agent.mdx +16 -8
  523. package/docs-site/zh/tutorials/nixos-managed-dind.mdx +160 -0
  524. package/docs-site/zh/tutorials/plugins.mdx +14 -0
  525. package/docs-site/zh/tutorials/rerun-and-cache.mdx +4 -2
  526. package/docs-site/zh/tutorials/sandbox-agent.mdx +17 -11
  527. package/docs-site/zh/tutorials/sandbox-providers.mdx +17 -13
  528. package/docs-site/zh/tutorials/sandbox-reuse.mdx +12 -19
  529. package/docs-site/zh/tutorials/select-and-preview.mdx +65 -0
  530. package/docs-site/zh/tutorials/verify-judge.mdx +78 -0
  531. package/docs-site/zh/tutorials/viewing-results.mdx +38 -1
  532. package/docs-site/zh/tutorials/write-experiment.mdx +7 -5
  533. package/docs-site/zh/tutorials/write-send.mdx +9 -8
  534. package/package.json +9 -5
  535. package/dist/report/built-in/overview.cjs +0 -44
  536. package/dist/report/built-in/overview.cjs.map +0 -1
  537. package/dist/report/built-in/overview.d.cts +0 -6
  538. package/dist/report/built-in/overview.d.mts +0 -6
  539. package/dist/report/built-in/overview.d.ts +0 -6
  540. package/dist/report/built-in/overview.js +0 -9
  541. package/dist/report/built-in/overview.js.map +0 -1
  542. package/dist/runner/gate-lease.cjs +0 -283
  543. package/dist/runner/gate-lease.cjs.map +0 -1
  544. package/dist/runner/gate-lease.d.cts +0 -62
  545. package/dist/runner/gate-lease.d.mts +0 -62
  546. package/dist/runner/gate-lease.d.ts +0 -62
  547. package/dist/runner/gate-lease.js +0 -22
  548. package/dist/runner/gate-lease.js.map +0 -1
@@ -71,7 +71,7 @@ langgraph/
71
71
  <div className="gd-file">
72
72
  <div className="gd-head"><span className="gd-name">{"niceeval.config.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+13"}</span></span></div>
73
73
  <div className="gd-body">
74
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineConfig } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 注:这个 app 的 .env 把标准的 OPENAI_API_KEY / OPENAI_BASE_URL 挪用给了 DeepSeek"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// (agent.py 里 ChatOpenAI 直接读这两个 env 名)。niceeval 的 judge(t.judge.autoevals.*)"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 不碰这两个名字:端点写在 judge.baseUrl,key 只读 NICEEVAL_JUDGE_KEY(或 judge.apiKeyEnv"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 指定的变量名),judge 走独立凭证,不和应用的模型配置互相干扰。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineConfig"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" name: { "}</span><span className="gdt2">{"\"zh-CN\""}</span><span className="gdt0">{": "}</span><span className="gdt2">{"\"LangGraph 示例\""}</span><span className="gdt0">{", en: "}</span><span className="gdt2">{"\"LangGraph example\""}</span><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" judge: { model: "}</span><span className="gdt2">{"\"gpt-5.4\""}</span><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" timeoutMs: "}</span><span className="gdt1">{"120_000"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 被测应用是用户自己起的长驻服务,别开太高并发。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" maxConcurrency: "}</span><span className="gdt1">{"2"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
74
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineConfig } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 注:这个 app 的 .env 把标准的 OPENAI_API_KEY / OPENAI_BASE_URL 挪用给了 DeepSeek"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// (agent.py 里 ChatOpenAI 直接读这两个 env 名)。niceeval 的 judge(t.judge.autoevals.*)"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 不碰这两个名字:端点写在 judge.baseUrl,并用 judge.apiKeyEnv 显式改读"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// NICEEVAL_JUDGE_KEY;judge 走独立凭证,不和应用的模型配置互相干扰。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineConfig"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" name: { "}</span><span className="gdt2">{"\"zh-CN\""}</span><span className="gdt0">{": "}</span><span className="gdt2">{"\"LangGraph 示例\""}</span><span className="gdt0">{", en: "}</span><span className="gdt2">{"\"LangGraph example\""}</span><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" judge: { model: "}</span><span className="gdt2">{"\"gpt-5.4\""}</span><span className="gdt0">{", apiKeyEnv: "}</span><span className="gdt2">{"\"NICEEVAL_JUDGE_KEY\""}</span><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" timeoutMs: "}</span><span className="gdt1">{"120_000"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 被测应用是用户自己起的长驻服务,别开太高并发。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" maxConcurrency: "}</span><span className="gdt1">{"2"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
75
75
  </div>
76
76
  </div>
77
77
 
@@ -92,7 +92,7 @@ langgraph/
92
92
  <div className="gd-file">
93
93
  <div className="gd-head"><span className="gd-name">{"evals/hitl-approve.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+24"}</span></span></div>
94
94
  <div className="gd-body">
95
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { equals, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// calculate 工具经 LangChain HumanInTheLoopMiddleware 挂了审批(见 agents/langgraph.ts、"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// origin src/backend/agent.py)。这条验证批准分支:approve 之后工具正常执行。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"//"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 提示词不提\"审批\"——不同模型看到提示词里的\"审批\"字样,有的会倾向于用文字反问用户"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// \"可以吗\"而不是真的发起工具调用(在 pi-sdk / claude-sdk 的接入里都复现过);审批门是"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 服务端中间件自动挂的,跟用户怎么问无关。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"HITL:calculate 经批准后正常执行\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"draft"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"用计算器算一下 (23+19)*3 等于多少\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(draft.status, "}</span><span className="gdt5">{"equals"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"waiting\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"requireInputRequest"}</span><span className="gdt0">{"({ action: "}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"approved"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"respond"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"approve\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" approved."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{", { status: "}</span><span className="gdt2">{"\"completed\""}</span><span className="gdt0">{" }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"messageIncludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"126"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"24"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
95
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { equals, pattern, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// calculate 工具经 LangChain HumanInTheLoopMiddleware 挂了审批(见 agents/langgraph.ts、"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// origin src/backend/agent.py)。这条验证批准分支:approve 之后工具正常执行。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"//"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 提示词不提\"审批\"——不同模型看到提示词里的\"审批\"字样,有的会倾向于用文字反问用户"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// \"可以吗\"而不是真的发起工具调用(在 pi-sdk / claude-sdk 的接入里都复现过);审批门是"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 服务端中间件自动挂的,跟用户怎么问无关。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"HITL:calculate 经批准后正常执行\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"draft"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"用计算器算一下 (23+19)*3 等于多少\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(draft.status, "}</span><span className="gdt5">{"equals"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"waiting\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"requireInputRequest"}</span><span className="gdt0">{"({ action: "}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"approved"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"respond"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"approve\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" approved."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{", { status: "}</span><span className="gdt2">{"\"completed\""}</span><span className="gdt0">{" }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(approved.message, "}</span><span className="gdt5">{"pattern"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"126"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"24"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
96
96
  </div>
97
97
  </div>
98
98
 
@@ -106,14 +106,14 @@ langgraph/
106
106
  <div className="gd-file">
107
107
  <div className="gd-head"><span className="gd-name">{"evals/session-isolation.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+19"}</span></span></div>
108
108
  <div className="gd-body">
109
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { includes, excludes } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 专门验证会话续接的两半承诺:同一条会话线里第二轮记得住第一轮说的名字"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// (LangGraph InMemorySaver 按 thread_id 续接成功);t.newSession() 造出的新会话线不共享历史。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试跨轮记忆与 newSession() 隔离\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫小明,帮我记住这个名字。\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"recall"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我刚才说我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" recall."}</span><span className="gdt5">{"messageIncludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(t.reply, "}</span><span className="gdt5">{"includes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"fresh"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"newSession"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" fresh."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(fresh.reply, "}</span><span className="gdt5">{"excludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
109
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { includes, excludes } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 专门验证会话续接的两半承诺:同一条会话线里第二轮记得住第一轮说的名字"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// (LangGraph InMemorySaver 按 thread_id 续接成功);t.newSession() 造出的新会话线不共享历史。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试跨轮记忆与 newSession() 隔离\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫小明,帮我记住这个名字。\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"recall"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我刚才说我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(recall.message, "}</span><span className="gdt5">{"includes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(t.reply, "}</span><span className="gdt5">{"includes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"fresh"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"newSession"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" fresh."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(fresh.reply, "}</span><span className="gdt5">{"excludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
110
110
  </div>
111
111
  </div>
112
112
 
113
113
  <div className="gd-file">
114
114
  <div className="gd-head"><span className="gd-name">{"evals/weather-tool.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+24"}</span></span></div>
115
115
  <div className="gd-body">
116
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { jsonMatch, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 验证 agent 遇到实时天气问题时调 get_weather,而不是直接编一个答案。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// get_weather 不是 gated 工具,它的 operation.started/operation.finished 完全来自 LangSmith span"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 派生,adapter 没有为它写一行帧映射(帧映射只补了 gated 的 calculate,见 agents/langgraph.ts)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" judge: "}</span><span className="gdt1">{"true"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试 agent 在天气问题中正确调用 get_weather 并基于结果作答\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"turn"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"北京今天天气怎么样?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" turn."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"()."}</span><span className="gdt5">{"orStop"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"group"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"调用 get_weather 且城市正确\""}</span><span className="gdt0">{", () "}</span><span className="gdt4">{"=>"}</span><span className="gdt0">{" {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"get_weather\""}</span><span className="gdt0">{", { input: "}</span><span className="gdt5">{"jsonMatch"}</span><span className="gdt0">{"({ city: "}</span><span className="gdt2">{"\"北京\""}</span><span className="gdt0">{" }) }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"messageIncludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"°C"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"气温"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"天气"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"晴"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"多云"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"雨"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"阴"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" turn.judge.autoevals"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"closedQA"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"助手是否给出了具体的天气数据(温度或天气状况),而不是拒绝回答或含糊其辞?\""}</span><span className="gdt0">{")"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"gate"}</span><span className="gdt0">{"("}</span><span className="gdt1">{"0.7"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"24"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
116
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { jsonMatch, pattern, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 验证 agent 遇到实时天气问题时调 get_weather,而不是直接编一个答案。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// get_weather 不是 gated 工具,它的 operation.started/operation.finished 完全来自 LangSmith span"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 派生,adapter 没有为它写一行帧映射(帧映射只补了 gated 的 calculate,见 agents/langgraph.ts)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" judge: "}</span><span className="gdt1">{"true"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试 agent 在天气问题中正确调用 get_weather 并基于结果作答\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"turn"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"北京今天天气怎么样?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" turn."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"()."}</span><span className="gdt5">{"orStop"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"group"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"调用 get_weather 且城市正确\""}</span><span className="gdt0">{", () "}</span><span className="gdt4">{"=>"}</span><span className="gdt0">{" {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"get_weather\""}</span><span className="gdt0">{", { input: "}</span><span className="gdt5">{"jsonMatch"}</span><span className="gdt0">{"({ city: "}</span><span className="gdt2">{"\"北京\""}</span><span className="gdt0">{" }) }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(turn.message, "}</span><span className="gdt5">{"pattern"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"°C"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"气温"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"天气"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"晴"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"多云"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"雨"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"阴"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" turn.judge.autoevals"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"closedQA"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"助手是否给出了具体的天气数据(温度或天气状况),而不是拒绝回答或含糊其辞?\""}</span><span className="gdt0">{")"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"gate"}</span><span className="gdt0">{"("}</span><span className="gdt1">{"0.7"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"24"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
117
117
  </div>
118
118
  </div>
119
119
 
@@ -20,7 +20,7 @@ Adapter 只需填写应用地址和审批接口地址。示例中的 `calculate`
20
20
 
21
21
  接入的全部代码变更(生成时从两个目录实测统计):
22
22
 
23
- <table className="gd-summary"><tbody><tr><th>{"类别"}</th><th>{"文件数"}</th><th>{"行数"}</th></tr><tr><td>{"应用侧配置(必要:依赖声明)"}</td><td>{"3"}</td><td>{"+7 −1"}</td></tr><tr><td>{"Adapter(必要:连接应用,协议映射由 NiceEval 提供)"}</td><td>{"2"}</td><td>{"+110"}</td></tr><tr><td>{"评估用例与实验(按需增长)"}</td><td>{"6"}</td><td>{"+123"}</td></tr><tr className="gd-total"><td>{"合计"}</td><td>{"11"}</td><td>{"+240 −1"}</td></tr></tbody></table>
23
+ <table className="gd-summary"><tbody><tr><th>{"类别"}</th><th>{"文件数"}</th><th>{"行数"}</th></tr><tr><td>{"应用侧配置(必要:依赖声明)"}</td><td>{"3"}</td><td>{"+7 −1"}</td></tr><tr><td>{"Adapter(必要:连接应用,协议映射由 NiceEval 提供)"}</td><td>{"2"}</td><td>{"+110"}</td></tr><tr><td>{"评估用例与实验(按需增长)"}</td><td>{"6"}</td><td>{"+122"}</td></tr><tr className="gd-total"><td>{"合计"}</td><td>{"11"}</td><td>{"+239 −1"}</td></tr></tbody></table>
24
24
 
25
25
  ## 文件清单
26
26
 
@@ -89,9 +89,9 @@ pi-sdk/
89
89
  </div>
90
90
 
91
91
  <div className="gd-file">
92
- <div className="gd-head"><span className="gd-name">{"evals/hitl-approve.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+25"}</span></span></div>
92
+ <div className="gd-head"><span className="gd-name">{"evals/hitl-approve.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+24"}</span></span></div>
93
93
  <div className="gd-body">
94
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { equals, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// calculate 工具经服务端 beforeToolCall 挂了审批(见 agents/pi-sdk.ts、origin src/backend/server.ts)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条验证批准分支:approve 之后工具正常执行,calledTool 的 status 是 \"completed\"。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"HITL:calculate 经批准后正常执行\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 提示词不提\"审批\"——一提审批,deepseek-v4-flash 就倾向于用文字问\"可以吗\",而不是真的发起"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 工具调用;审批门是服务端 beforeToolCall 自动挂的,跟用户怎么问无关,越自然越准确触发工具。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"draft"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"用计算器算一下 (23+19)*3 等于多少\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// t.parked() 之类的作用域断言是延迟求值的(评到 run 结束时的最终状态),这一轮结束后马上"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// continue 到 approve,run 早就不再\"停着\"了——判\"当下有没有停在审批上\"只能看这个"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// TurnHandle 自己的 status,不能用 t.parked()。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(draft.status, "}</span><span className="gdt5">{"equals"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"waiting\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"requireInputRequest"}</span><span className="gdt0">{"({ action: "}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"approved"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"respond"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"approve\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" approved."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{", { status: "}</span><span className="gdt2">{"\"completed\""}</span><span className="gdt0">{" }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"messageIncludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"126"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"24"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"25"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
94
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { equals, pattern, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// calculate 工具经服务端 beforeToolCall 挂了审批(见 agents/pi-sdk.ts、origin src/backend/server.ts)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条验证批准分支:approve 之后工具正常执行,calledTool 的 status 是 \"completed\"。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"HITL:calculate 经批准后正常执行\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 提示词不提\"审批\"——一提审批,deepseek-v4-flash 就倾向于用文字问\"可以吗\",而不是真的发起"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 工具调用;审批门是服务端 beforeToolCall 自动挂的,跟用户怎么问无关,越自然越准确触发工具。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"draft"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"用计算器算一下 (23+19)*3 等于多少\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 作用域断言在 Run 封口时才读取最终状态;继续到 approve 后已经不再等待。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 要判断当前是否停在审批上,只能读取这个 TurnHandle 自己的 status。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(draft.status, "}</span><span className="gdt5">{"equals"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"waiting\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"requireInputRequest"}</span><span className="gdt0">{"({ action: "}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"approved"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"respond"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"approve\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" approved."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"calculate\""}</span><span className="gdt0">{", { status: "}</span><span className="gdt2">{"\"completed\""}</span><span className="gdt0">{" }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(approved.message, "}</span><span className="gdt5">{"pattern"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"126"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"24"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
95
95
  </div>
96
96
  </div>
97
97
 
@@ -105,14 +105,14 @@ pi-sdk/
105
105
  <div className="gd-file">
106
106
  <div className="gd-head"><span className="gd-name">{"evals/session-isolation.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+23"}</span></span></div>
107
107
  <div className="gd-body">
108
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { includes, excludes } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 专门验证 ctx.session 续接的两半承诺:同一条会话线里第二轮记得住第一轮说的名字"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// (server.ts 内存 Map<sessionId, AgentMessage[]> 续接成功);t.newSession() 开出的新会话线"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 拿到一份全新的 ctx.session,不共享历史(常见 bug:adapter 把 sessionId 存进模块级变量而不是"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 读 ctx.session.id,新会话线会被错误地续上旧历史,隔离会静默失真且不报错)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试跨轮记忆与 newSession() 隔离\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫小明,帮我记住这个名字。\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"recall"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我刚才说我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" recall."}</span><span className="gdt5">{"messageIncludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(t.reply, "}</span><span className="gdt5">{"includes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"fresh"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"newSession"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" fresh."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 新 session 没有历史,不应该知道\"小明\"这个事实——用 fresh.* 只看这条新 session 自己的事件,"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 不是 t.*(t.* 是 run 级聚合,会同时看到主 session 的事件)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(fresh.reply, "}</span><span className="gdt5">{"excludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
108
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { includes, excludes } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 专门验证 ctx.session 续接的两半承诺:同一条会话线里第二轮记得住第一轮说的名字"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// (server.ts 内存 Map<sessionId, AgentMessage[]> 续接成功);t.newSession() 开出的新会话线"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 拿到一份全新的 ctx.session,不共享历史(常见 bug:adapter 把 sessionId 存进模块级变量而不是"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 读 ctx.session.id,新会话线会被错误地续上旧历史,隔离会静默失真且不报错)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试跨轮记忆与 newSession() 隔离\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫小明,帮我记住这个名字。\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"recall"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我刚才说我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(recall.message, "}</span><span className="gdt5">{"includes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(t.reply, "}</span><span className="gdt5">{"includes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"fresh"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"newSession"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" fresh."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"我叫什么名字?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 新 session 没有历史,不应该知道\"小明\"这个事实——用 fresh.* 只看这条新 session 自己的事件,"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 不是 t.*(t.* 是 run 级聚合,会同时看到主 session 的事件)。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(fresh.reply, "}</span><span className="gdt5">{"excludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"小明\""}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
109
109
  </div>
110
110
  </div>
111
111
 
112
112
  <div className="gd-file">
113
113
  <div className="gd-head"><span className="gd-name">{"evals/weather-tool.eval.ts"}</span><span className="gd-stats"><span className="gd-plus">{"+23"}</span></span></div>
114
114
  <div className="gd-body">
115
- <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { jsonMatch, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 验证 agent 遇到实时天气问题时会调 get_weather,而不是直接编一个答案。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" judge: "}</span><span className="gdt1">{"true"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试 agent 在天气问题中正确调用 get_weather 并基于结果作答\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"turn"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"北京今天天气怎么样?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" turn."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"()."}</span><span className="gdt5">{"orStop"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"group"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"调用 get_weather 且城市正确\""}</span><span className="gdt0">{", () "}</span><span className="gdt4">{"=>"}</span><span className="gdt0">{" {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"get_weather\""}</span><span className="gdt0">{", { input: "}</span><span className="gdt5">{"jsonMatch"}</span><span className="gdt0">{"({ city: "}</span><span className="gdt2">{"\"北京\""}</span><span className="gdt0">{" }) }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"messageIncludes"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"°C"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"气温"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"天气"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"晴"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"多云"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"雨"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"阴"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 「是否走了工具」由上面的 t.calledTool 确定性把关;judge 只评回复本身的质量。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" turn.judge.autoevals"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"closedQA"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"助手是否给出了具体的天气数据(温度或天气状况),而不是拒绝回答或含糊其辞?\""}</span><span className="gdt0">{")"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"gate"}</span><span className="gdt0">{"("}</span><span className="gdt1">{"0.7"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
115
+ <table className="gd-table"><tbody><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"1"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { defineEval } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"2"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"import"}</span><span className="gdt0">{" { jsonMatch, pattern, toolMatch } "}</span><span className="gdt4">{"from"}</span><span className="gdt0">{" "}</span><span className="gdt2">{"\"niceeval/expect\""}</span><span className="gdt0">{";"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"3"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"4"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt6">{"// 这条 eval 验证 agent 遇到实时天气问题时会调 get_weather,而不是直接编一个答案。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"5"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt4">{"export"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"default"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"defineEval"}</span><span className="gdt0">{"({"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"6"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" judge: "}</span><span className="gdt1">{"true"}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"7"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" description: "}</span><span className="gdt2">{"\"测试 agent 在天气问题中正确调用 get_weather 并基于结果作答\""}</span><span className="gdt0">{","}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"8"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"9"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"async"}</span><span className="gdt0">{" "}</span><span className="gdt5">{"test"}</span><span className="gdt0">{"("}</span><span className="gdt7">{"t"}</span><span className="gdt0">{") {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"10"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"const"}</span><span className="gdt0">{" "}</span><span className="gdt1">{"turn"}</span><span className="gdt0">{" "}</span><span className="gdt4">{"="}</span><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"send"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"北京今天天气怎么样?\""}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"11"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" turn."}</span><span className="gdt5">{"succeeded"}</span><span className="gdt0">{"()."}</span><span className="gdt5">{"orStop"}</span><span className="gdt0">{"();"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"12"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"13"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt4">{"await"}</span><span className="gdt0">{" t."}</span><span className="gdt5">{"group"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"调用 get_weather 且城市正确\""}</span><span className="gdt0">{", () "}</span><span className="gdt4">{"=>"}</span><span className="gdt0">{" {"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"14"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"calledTool"}</span><span className="gdt0">{"("}</span><span className="gdt5">{"toolMatch"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"get_weather\""}</span><span className="gdt0">{", { input: "}</span><span className="gdt5">{"jsonMatch"}</span><span className="gdt0">{"({ city: "}</span><span className="gdt2">{"\"北京\""}</span><span className="gdt0">{" }) }));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"15"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" t."}</span><span className="gdt5">{"check"}</span><span className="gdt0">{"(turn.message, "}</span><span className="gdt5">{"pattern"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"/"}</span><span className="gdt8">{"°C"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"气温"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"天气"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"晴"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"多云"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"雨"}</span><span className="gdt4">{"|"}</span><span className="gdt8">{"阴"}</span><span className="gdt2">{"/"}</span><span className="gdt0">{"));"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"16"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" });"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"17"}</td><td className="gd-sign">{"+"}</td><td className="gd-code">{" "}</td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"18"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" "}</span><span className="gdt6">{"// 「是否走了工具」由上面的 t.calledTool 确定性把关;judge 只评回复本身的质量。"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"19"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" turn.judge.autoevals"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"20"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"closedQA"}</span><span className="gdt0">{"("}</span><span className="gdt2">{"\"助手是否给出了具体的天气数据(温度或天气状况),而不是拒绝回答或含糊其辞?\""}</span><span className="gdt0">{")"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"21"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" ."}</span><span className="gdt5">{"gate"}</span><span className="gdt0">{"("}</span><span className="gdt1">{"0.7"}</span><span className="gdt0">{");"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"22"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{" },"}</span></td></tr><tr className="gd-add"><td className="gd-ln"></td><td className="gd-ln">{"23"}</td><td className="gd-sign">{"+"}</td><td className="gd-code"><span className="gdt0">{"});"}</span></td></tr></tbody></table>
116
116
  </div>
117
117
  </div>
118
118
 
@@ -282,12 +282,12 @@ turn.data.amount; // 类型是 number——不是 unknown,不用手动收窄
282
282
 
283
283
  | Turn 字段 | 运行器拿它做什么 | 评估用例作者在哪感知到 |
284
284
  |---|---|---|
285
- | `events` | 追加进本会话线的事件流,`deriveRunFacts` 折叠成事实(`toolCalls`、`parked`、`messageCount`…)。最后一条 assistant `message` 同时更新 `t.reply` | 全部作用域断言:`t.calledTool()`、`t.messageIncludes()`…。`t.events` 可直接查 |
286
- | `status` | 记录本轮终态。`"waiting"` 时把本轮的 `input.requested` 收进待答列表 | `t.succeeded()` / `t.parked()`。`"waiting"` 之后用 `t.respond()` 接 |
285
+ | `events` | 追加进本会话线,归一为消息与逻辑工具 occurrence。最后一条 assistant `message` 同时更新 `t.reply` | `turn.calledTool()`、`turn.event(...)` 等作用域断言;`t.events` 可直接查 |
286
+ | `status` | 记录本轮终态。`"waiting"` 时把本轮的 `input.requested` 收进待答列表 | `t.succeeded()` `t.check(turn.status, equals("waiting"))`。`"waiting"` 之后用 `t.respond()` 接 |
287
287
  | `usage` | 逐轮累加到会话线与整次运行 | `maxTokens` / `maxCost` 断言、报告里的用量 |
288
- | `data` | 按 send 上声明的 `output` schema 校验——不匹配本轮直接 `failed`。通过才保存 | `turn.data`(按声明强类型)、`outputEquals` |
288
+ | `data` | 按 send 上声明的 `output` schema 校验——不匹配本轮直接 `failed`。通过才保存 | `turn.data`(按声明强类型)、`t.check(turn.data, equals(...))` 或 `matches(...)` |
289
289
 
290
- `await t.send()` 拿到的句柄就是对这一轮 Turn 的视角:turn 级断言(`draft.parked()` 这类)只看这一轮的 `events`,`t` 级断言看整条会话线累计的。所以 Adapter 不需要"配合"任何断言——把这四个字段填对,断言、评分、报告全部在下游自动发生。
290
+ `await t.send()` 拿到的句柄就是对这一轮 Turn 的视角:turn 级断言只看这一轮的 `events` 与字段,`t` 级断言看整条会话线累计的。因此 Adapter 不需要"配合"任何断言——把这四个字段填对,断言、评分、报告全部在下游自动发生。
291
291
 
292
292
  ## 能力从哪来:构造证明,不是问卷
293
293
 
@@ -300,7 +300,7 @@ Sandbox Agent,只有它的回调拿到带真实 `sandbox` 的 `SandboxAgentCon
300
300
  |---|---|---|
301
301
  | `t.send`、`t.sendFile`、`t.check`、`t.judge`… | 任何 Agent 都有 | 无 |
302
302
  | 多次 `t.send()`、`t.reply`、`t.newSession()` | `send` 里接了 `ctx.session` 的续接存取器(typed slot 或 `id` + `capture()`)——没接则每轮各是新对话 | 无——存取器就在 `ctx` 上 |
303
- | `t.respond()` / `t.parked()` 等 HITL | send 返回过 `"waiting"` + `input.requested` 事件 | 无——做到了就是有 |
303
+ | `t.respond()`、状态检查等 HITL | send 返回过 `"waiting"` + `input.requested` 事件 | 无——做到了就是有 |
304
304
  | `turn.calledTool()` / `turn.toolOrder()` 等正断言 | events 里有配对的 `operation.started` / `operation.finished` | 无——有事件就能断 |
305
305
  | `t.notCalledTool()` / `t.usedNoTools()` 等负断言**可信** | 事件来自带完整性证明的官方转换器 | 无——证明随转换器返回值走 |
306
306
  | `t.sandbox`、`t.sandbox.fileChanged()` 等 | `defineSandboxAgent` 构造 | 无 |