niceeval 0.15.1-canary.76 → 0.16.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (714) hide show
  1. package/INDEX.md +45 -40
  2. package/dist/adapter-execution-trace.cjs.map +1 -1
  3. package/dist/adapter-execution-trace.d.cts +26 -0
  4. package/dist/adapter-execution-trace.d.mts +26 -0
  5. package/dist/adapter-execution-trace.d.ts +26 -0
  6. package/dist/adapter-usage.cjs.map +1 -1
  7. package/dist/adapter-usage.d.cts +9 -0
  8. package/dist/adapter-usage.d.mts +9 -0
  9. package/dist/adapter-usage.d.ts +9 -0
  10. package/dist/adapter.cjs +19 -4
  11. package/dist/adapter.cjs.map +1 -1
  12. package/dist/adapter.d.cts +37 -12
  13. package/dist/adapter.d.mts +37 -12
  14. package/dist/adapter.d.ts +37 -12
  15. package/dist/agents/ai-sdk.cjs +3 -0
  16. package/dist/agents/ai-sdk.cjs.map +1 -1
  17. package/dist/agents/bub.cjs +3 -0
  18. package/dist/agents/bub.cjs.map +1 -1
  19. package/dist/agents/claude-code-native.cjs +28 -2
  20. package/dist/agents/claude-code-native.cjs.map +1 -1
  21. package/dist/agents/claude-code.cjs +3 -0
  22. package/dist/agents/claude-code.cjs.map +1 -1
  23. package/dist/agents/codex.cjs +3 -0
  24. package/dist/agents/codex.cjs.map +1 -1
  25. package/dist/agents/coding-cli-versions.cjs +9 -10
  26. package/dist/agents/coding-cli-versions.cjs.map +1 -1
  27. package/dist/agents/coding-cli-versions.d.cts +1 -1
  28. package/dist/agents/coding-cli-versions.d.mts +1 -1
  29. package/dist/agents/coding-cli-versions.d.ts +1 -1
  30. package/dist/agents/deepseek-harness.cjs +3 -0
  31. package/dist/agents/deepseek-harness.cjs.map +1 -1
  32. package/dist/agents/hermes.cjs +3 -0
  33. package/dist/agents/hermes.cjs.map +1 -1
  34. package/dist/agents/index.cjs.map +1 -1
  35. package/dist/agents/index.d.cts +3 -2
  36. package/dist/agents/index.d.mts +3 -2
  37. package/dist/agents/index.d.ts +3 -2
  38. package/dist/agents/omp.cjs +3 -0
  39. package/dist/agents/omp.cjs.map +1 -1
  40. package/dist/agents/openai-compat.cjs +6 -0
  41. package/dist/agents/openai-compat.cjs.map +1 -1
  42. package/dist/agents/openclaw.cjs +3 -0
  43. package/dist/agents/openclaw.cjs.map +1 -1
  44. package/dist/agents/opencode.cjs +3 -0
  45. package/dist/agents/opencode.cjs.map +1 -1
  46. package/dist/agents/sdk-streams.cjs +9 -0
  47. package/dist/agents/sdk-streams.cjs.map +1 -1
  48. package/dist/agents/ui-message-stream.cjs +3 -0
  49. package/dist/agents/ui-message-stream.cjs.map +1 -1
  50. package/dist/assertions/api.cjs.map +1 -1
  51. package/dist/assertions/api.d.cts +27 -5
  52. package/dist/assertions/api.d.mts +27 -5
  53. package/dist/assertions/api.d.ts +27 -5
  54. package/dist/assertions/collection.cjs +6 -0
  55. package/dist/assertions/collection.cjs.map +1 -1
  56. package/dist/assertions/collection.d.cts +4 -2
  57. package/dist/assertions/collection.d.mts +4 -2
  58. package/dist/assertions/collection.d.ts +4 -2
  59. package/dist/assertions/collection.js +12 -8
  60. package/dist/assertions/context-match.cjs +70 -0
  61. package/dist/assertions/context-match.cjs.map +1 -0
  62. package/dist/assertions/context-match.d.cts +107 -0
  63. package/dist/assertions/context-match.d.mts +107 -0
  64. package/dist/assertions/context-match.d.ts +107 -0
  65. package/dist/assertions/context-match.js +12 -0
  66. package/dist/assertions/context-match.js.map +1 -0
  67. package/dist/assertions/full-content.cjs +109 -0
  68. package/dist/assertions/full-content.cjs.map +1 -0
  69. package/dist/assertions/full-content.d.cts +6 -0
  70. package/dist/assertions/full-content.d.mts +6 -0
  71. package/dist/assertions/full-content.d.ts +6 -0
  72. package/dist/assertions/full-content.js +12 -0
  73. package/dist/assertions/full-content.js.map +1 -0
  74. package/dist/assertions/judge-presets.cjs +2 -34
  75. package/dist/assertions/judge-presets.cjs.map +1 -1
  76. package/dist/assertions/judge-presets.d.cts +6 -9
  77. package/dist/assertions/judge-presets.d.mts +6 -9
  78. package/dist/assertions/judge-presets.d.ts +6 -9
  79. package/dist/assertions/judge-presets.js +8 -8
  80. package/dist/assertions/judge.cjs +107 -29
  81. package/dist/assertions/judge.cjs.map +1 -1
  82. package/dist/assertions/judge.d.cts +2 -0
  83. package/dist/assertions/judge.d.mts +2 -0
  84. package/dist/assertions/judge.d.ts +2 -0
  85. package/dist/assertions/match.cjs +244 -5
  86. package/dist/assertions/match.cjs.map +1 -1
  87. package/dist/assertions/match.d.cts +43 -3
  88. package/dist/assertions/match.d.mts +43 -3
  89. package/dist/assertions/match.d.ts +43 -3
  90. package/dist/assertions/match.js +88 -78
  91. package/dist/assertions/material.cjs +520 -0
  92. package/dist/assertions/material.cjs.map +1 -0
  93. package/dist/assertions/material.d.cts +56 -0
  94. package/dist/assertions/material.d.mts +56 -0
  95. package/dist/assertions/material.d.ts +56 -0
  96. package/dist/assertions/material.js +20 -0
  97. package/dist/assertions/material.js.map +1 -0
  98. package/dist/assertions/record/attachment.cjs +5 -2
  99. package/dist/assertions/record/attachment.cjs.map +1 -1
  100. package/dist/assertions/record/codec.cjs +2 -1
  101. package/dist/assertions/record/codec.cjs.map +1 -1
  102. package/dist/assertions/record/model.cjs.map +1 -1
  103. package/dist/assertions/record/model.d.cts +6 -1
  104. package/dist/assertions/record/model.d.mts +6 -1
  105. package/dist/assertions/record/model.d.ts +6 -1
  106. package/dist/assertions/runtime.cjs +207 -14
  107. package/dist/assertions/runtime.cjs.map +1 -1
  108. package/dist/assertions/runtime.d.cts +9 -0
  109. package/dist/assertions/runtime.d.mts +9 -0
  110. package/dist/assertions/runtime.d.ts +9 -0
  111. package/dist/assertions/runtime.js +14 -8
  112. package/dist/assertions/score-match-audit.cjs +18 -8
  113. package/dist/assertions/score-match-audit.cjs.map +1 -1
  114. package/dist/assertions/score-match-gateway.cjs +83 -25
  115. package/dist/assertions/score-match-gateway.cjs.map +1 -1
  116. package/dist/assertions/score-match-gateway.d.cts +2 -0
  117. package/dist/assertions/score-match-gateway.d.mts +2 -0
  118. package/dist/assertions/score-match-gateway.d.ts +2 -0
  119. package/dist/assertions/value-composition.test-d.cjs +77 -0
  120. package/dist/assertions/value-composition.test-d.cjs.map +1 -0
  121. package/dist/assertions/value-composition.test-d.d.cts +1 -0
  122. package/dist/assertions/value-composition.test-d.d.mts +1 -0
  123. package/dist/assertions/value-composition.test-d.d.ts +1 -0
  124. package/dist/assertions/value-composition.test-d.js +4 -0
  125. package/dist/assertions/value-composition.test-d.js.map +1 -0
  126. package/dist/context/assert-first.cjs +207 -213
  127. package/dist/context/assert-first.cjs.map +1 -1
  128. package/dist/context/assert-first.d.cts +49 -23
  129. package/dist/context/assert-first.d.mts +49 -23
  130. package/dist/context/assert-first.d.ts +49 -23
  131. package/dist/context/session.cjs +111 -9
  132. package/dist/context/session.cjs.map +1 -1
  133. package/dist/context/session.d.cts +18 -0
  134. package/dist/context/session.d.mts +18 -0
  135. package/dist/context/session.d.ts +18 -0
  136. package/dist/define.cjs +17 -1
  137. package/dist/define.cjs.map +1 -1
  138. package/dist/eval/cli/contribution.cjs +2 -0
  139. package/dist/eval/cli/contribution.cjs.map +1 -1
  140. package/dist/eval/host/index.cjs +2 -0
  141. package/dist/eval/host/index.cjs.map +1 -1
  142. package/dist/eval/record/evaluation.cjs +3 -0
  143. package/dist/eval/record/evaluation.cjs.map +1 -1
  144. package/dist/eval/record/membership-provenance.d.cts +15 -15
  145. package/dist/eval/record/membership-provenance.d.mts +15 -15
  146. package/dist/eval/record/membership-provenance.d.ts +15 -15
  147. package/dist/eval/record/score.cjs +7 -0
  148. package/dist/eval/record/score.cjs.map +1 -1
  149. package/dist/eval/record/sealed-assertion.cjs +3 -0
  150. package/dist/eval/record/sealed-assertion.cjs.map +1 -1
  151. package/dist/eval/record/verdict.cjs +7 -0
  152. package/dist/eval/record/verdict.cjs.map +1 -1
  153. package/dist/expect/index.cjs +8 -2
  154. package/dist/expect/index.cjs.map +1 -1
  155. package/dist/expect/index.d.cts +9 -4
  156. package/dist/expect/index.d.mts +9 -4
  157. package/dist/expect/index.d.ts +9 -4
  158. package/dist/expect/index.js +72 -62
  159. package/dist/expect/index.mjs +72 -62
  160. package/dist/experiment/host/cli/contribution.cjs +37 -23
  161. package/dist/experiment/host/cli/contribution.cjs.map +1 -1
  162. package/dist/experiment/host/cli/contribution.d.cts +9 -1
  163. package/dist/experiment/host/cli/contribution.d.mts +9 -1
  164. package/dist/experiment/host/cli/contribution.d.ts +9 -1
  165. package/dist/experiment/host/current.cjs +74 -0
  166. package/dist/experiment/host/current.cjs.map +1 -0
  167. package/dist/experiment/host/current.d.cts +33 -0
  168. package/dist/experiment/host/current.d.mts +33 -0
  169. package/dist/experiment/host/current.d.ts +33 -0
  170. package/dist/experiment/host/current.js +10 -0
  171. package/dist/experiment/host/current.js.map +1 -0
  172. package/dist/experiment/host/index.cjs +17 -174
  173. package/dist/experiment/host/index.cjs.map +1 -1
  174. package/dist/experiment/host/index.d.cts +8 -54
  175. package/dist/experiment/host/index.d.mts +8 -54
  176. package/dist/experiment/host/index.d.ts +8 -54
  177. package/dist/experiment/host/operations.cjs +33 -10
  178. package/dist/experiment/host/operations.cjs.map +1 -1
  179. package/dist/experiment/host/operations.d.cts +29 -1
  180. package/dist/experiment/host/operations.d.mts +29 -1
  181. package/dist/experiment/host/operations.d.ts +29 -1
  182. package/dist/experiment/host/operations.js +6 -4
  183. package/dist/experiment/host/requirements.cjs +3 -0
  184. package/dist/experiment/host/requirements.cjs.map +1 -0
  185. package/dist/experiment/host/requirements.d.cts +3 -0
  186. package/dist/experiment/host/requirements.d.mts +3 -0
  187. package/dist/experiment/host/requirements.d.ts +3 -0
  188. package/dist/experiment/host/requirements.js +4 -0
  189. package/dist/experiment/host/requirements.js.map +1 -0
  190. package/dist/experiment/host/runtime.cjs +179 -0
  191. package/dist/experiment/host/runtime.cjs.map +1 -0
  192. package/dist/experiment/host/runtime.d.cts +79 -0
  193. package/dist/experiment/host/runtime.d.mts +79 -0
  194. package/dist/experiment/host/runtime.d.ts +79 -0
  195. package/dist/experiment/host/runtime.js +6 -0
  196. package/dist/experiment/host/runtime.js.map +1 -0
  197. package/dist/experiment/host/types.cjs.map +1 -1
  198. package/dist/experiment/host/types.d.cts +17 -16
  199. package/dist/experiment/host/types.d.mts +17 -16
  200. package/dist/experiment/host/types.d.ts +17 -16
  201. package/dist/fresh-import.cjs +2 -1
  202. package/dist/fresh-import.cjs.map +1 -1
  203. package/dist/index.cjs +4 -2
  204. package/dist/index.cjs.map +1 -1
  205. package/dist/index.d.cts +12 -6
  206. package/dist/index.d.mts +12 -6
  207. package/dist/index.d.ts +12 -6
  208. package/dist/index.js +32 -30
  209. package/dist/index.mjs +32 -30
  210. package/dist/inspection/artifacts.cjs +83 -54
  211. package/dist/inspection/artifacts.cjs.map +1 -1
  212. package/dist/inspection/artifacts.d.cts +8 -0
  213. package/dist/inspection/artifacts.d.mts +8 -0
  214. package/dist/inspection/artifacts.d.ts +8 -0
  215. package/dist/inspection/artifacts.js +2 -0
  216. package/dist/inspection/assertions.cjs +4 -1
  217. package/dist/inspection/assertions.cjs.map +1 -1
  218. package/dist/inspection/cli/contribution.cjs +27 -4
  219. package/dist/inspection/cli/contribution.cjs.map +1 -1
  220. package/dist/inspection/cli/contribution.d.cts +2 -1
  221. package/dist/inspection/cli/contribution.d.mts +2 -1
  222. package/dist/inspection/cli/contribution.d.ts +2 -1
  223. package/dist/inspection/execution-traces.cjs +114 -8
  224. package/dist/inspection/execution-traces.cjs.map +1 -1
  225. package/dist/inspection/execution-traces.d.cts +38 -0
  226. package/dist/inspection/execution-traces.d.mts +38 -0
  227. package/dist/inspection/execution-traces.d.ts +38 -0
  228. package/dist/inspection/facts.cjs +12 -0
  229. package/dist/inspection/facts.cjs.map +1 -1
  230. package/dist/inspection/limits.cjs +5 -1
  231. package/dist/inspection/limits.cjs.map +1 -1
  232. package/dist/inspection/limits.d.cts +4 -0
  233. package/dist/inspection/limits.d.mts +4 -0
  234. package/dist/inspection/limits.d.ts +4 -0
  235. package/dist/inspection/limits.js +6 -2
  236. package/dist/inspection/overview.cjs +74 -33
  237. package/dist/inspection/overview.cjs.map +1 -1
  238. package/dist/inspection/overview.d.cts +19 -1
  239. package/dist/inspection/overview.d.mts +19 -1
  240. package/dist/inspection/overview.d.ts +19 -1
  241. package/dist/inspection/overview.js +4 -2
  242. package/dist/inspection/project-input.cjs +9 -0
  243. package/dist/inspection/project-input.cjs.map +1 -0
  244. package/dist/inspection/project-input.d.cts +20 -0
  245. package/dist/inspection/project-input.d.mts +20 -0
  246. package/dist/inspection/project-input.d.ts +20 -0
  247. package/dist/inspection/project-input.js +8 -0
  248. package/dist/inspection/project-input.js.map +1 -0
  249. package/dist/inspection/project-result.cjs +60 -0
  250. package/dist/inspection/project-result.cjs.map +1 -0
  251. package/dist/inspection/project-result.d.cts +1264 -0
  252. package/dist/inspection/project-result.d.mts +1264 -0
  253. package/dist/inspection/project-result.d.ts +1264 -0
  254. package/dist/inspection/project-result.js +10 -0
  255. package/dist/inspection/project-result.js.map +1 -0
  256. package/dist/inspection/project-source.cjs +109 -0
  257. package/dist/inspection/project-source.cjs.map +1 -0
  258. package/dist/inspection/project-source.d.cts +10 -0
  259. package/dist/inspection/project-source.d.mts +10 -0
  260. package/dist/inspection/project-source.d.ts +10 -0
  261. package/dist/inspection/project-source.js +6 -0
  262. package/dist/inspection/project-source.js.map +1 -0
  263. package/dist/inspection/project.cjs +100 -0
  264. package/dist/inspection/project.cjs.map +1 -0
  265. package/dist/inspection/project.d.cts +3 -0
  266. package/dist/inspection/project.d.mts +3 -0
  267. package/dist/inspection/project.d.ts +3 -0
  268. package/dist/inspection/project.js +6 -0
  269. package/dist/inspection/project.js.map +1 -0
  270. package/dist/inspection/protocol.cjs +19 -9
  271. package/dist/inspection/protocol.cjs.map +1 -1
  272. package/dist/inspection/protocol.d.cts +5506 -38189
  273. package/dist/inspection/protocol.d.mts +5506 -38189
  274. package/dist/inspection/protocol.d.ts +5506 -38189
  275. package/dist/inspection/public.mjs +896 -84
  276. package/dist/inspection/query.cjs +2 -0
  277. package/dist/inspection/query.cjs.map +1 -1
  278. package/dist/inspection/results.cjs +234 -29
  279. package/dist/inspection/results.cjs.map +1 -1
  280. package/dist/inspection/results.d.cts +870 -13
  281. package/dist/inspection/results.d.mts +870 -13
  282. package/dist/inspection/results.d.ts +870 -13
  283. package/dist/inspection/results.js +36 -34
  284. package/dist/inspection/select.cjs +126 -14
  285. package/dist/inspection/select.cjs.map +1 -1
  286. package/dist/inspection/select.d.cts +1 -1
  287. package/dist/inspection/select.d.mts +1 -1
  288. package/dist/inspection/select.d.ts +1 -1
  289. package/dist/inspection/source.cjs +4 -0
  290. package/dist/inspection/source.cjs.map +1 -1
  291. package/dist/inspection/sources.cjs +5 -0
  292. package/dist/inspection/sources.cjs.map +1 -1
  293. package/dist/inspection/trace.cjs +82 -84
  294. package/dist/inspection/trace.cjs.map +1 -1
  295. package/dist/inspection/trace.d.cts +7 -2
  296. package/dist/inspection/trace.d.mts +7 -2
  297. package/dist/inspection/trace.d.ts +7 -2
  298. package/dist/model-slots.cjs +35 -0
  299. package/dist/model-slots.cjs.map +1 -0
  300. package/dist/model-slots.d.cts +19 -0
  301. package/dist/model-slots.d.mts +19 -0
  302. package/dist/model-slots.d.ts +19 -0
  303. package/dist/model-slots.js +8 -0
  304. package/dist/model-slots.js.map +1 -0
  305. package/dist/o11y/adapter-call-price.cjs +4 -0
  306. package/dist/o11y/adapter-call-price.cjs.map +1 -1
  307. package/dist/o11y/adapter-call-price.d.cts +24 -0
  308. package/dist/o11y/adapter-call-price.d.mts +24 -0
  309. package/dist/o11y/adapter-call-price.d.ts +24 -0
  310. package/dist/o11y/adapter-call-price.js +6 -2
  311. package/dist/o11y/adapter-usage-projection.cjs +158 -0
  312. package/dist/o11y/adapter-usage-projection.cjs.map +1 -0
  313. package/dist/o11y/adapter-usage-projection.d.cts +141 -0
  314. package/dist/o11y/adapter-usage-projection.d.mts +141 -0
  315. package/dist/o11y/adapter-usage-projection.d.ts +141 -0
  316. package/dist/o11y/adapter-usage-projection.js +20 -0
  317. package/dist/o11y/adapter-usage-projection.js.map +1 -0
  318. package/dist/o11y/eval-usage.cjs +149 -0
  319. package/dist/o11y/eval-usage.cjs.map +1 -0
  320. package/dist/o11y/eval-usage.d.cts +46 -0
  321. package/dist/o11y/eval-usage.d.mts +46 -0
  322. package/dist/o11y/eval-usage.d.ts +46 -0
  323. package/dist/o11y/eval-usage.js +8 -0
  324. package/dist/o11y/eval-usage.js.map +1 -0
  325. package/dist/o11y/judge-call-price.cjs +62 -0
  326. package/dist/o11y/judge-call-price.cjs.map +1 -0
  327. package/dist/o11y/judge-call-price.d.cts +3 -0
  328. package/dist/o11y/judge-call-price.d.mts +3 -0
  329. package/dist/o11y/judge-call-price.d.ts +3 -0
  330. package/dist/o11y/judge-call-price.js +6 -0
  331. package/dist/o11y/judge-call-price.js.map +1 -0
  332. package/dist/o11y/judge-usage-projection.cjs +134 -0
  333. package/dist/o11y/judge-usage-projection.cjs.map +1 -0
  334. package/dist/o11y/judge-usage-projection.d.cts +18 -0
  335. package/dist/o11y/judge-usage-projection.d.mts +18 -0
  336. package/dist/o11y/judge-usage-projection.d.ts +18 -0
  337. package/dist/o11y/judge-usage-projection.js +20 -0
  338. package/dist/o11y/judge-usage-projection.js.map +1 -0
  339. package/dist/o11y/judge-usage-receipt.cjs +135 -0
  340. package/dist/o11y/judge-usage-receipt.cjs.map +1 -0
  341. package/dist/o11y/judge-usage-receipt.d.cts +5 -0
  342. package/dist/o11y/judge-usage-receipt.d.mts +5 -0
  343. package/dist/o11y/judge-usage-receipt.d.ts +5 -0
  344. package/dist/o11y/judge-usage-receipt.js +6 -0
  345. package/dist/o11y/judge-usage-receipt.js.map +1 -0
  346. package/dist/o11y/judge-usage.cjs +171 -0
  347. package/dist/o11y/judge-usage.cjs.map +1 -0
  348. package/dist/o11y/judge-usage.d.cts +35 -0
  349. package/dist/o11y/judge-usage.d.mts +35 -0
  350. package/dist/o11y/judge-usage.d.ts +35 -0
  351. package/dist/o11y/judge-usage.js +6 -0
  352. package/dist/o11y/judge-usage.js.map +1 -0
  353. package/dist/plugin/contracts.cjs +2 -0
  354. package/dist/plugin/contracts.cjs.map +1 -1
  355. package/dist/plugin/link.cjs +7 -0
  356. package/dist/plugin/link.cjs.map +1 -1
  357. package/dist/record/attachment/protocol.cjs +8 -6
  358. package/dist/record/attachment/protocol.cjs.map +1 -1
  359. package/dist/record/attachment/protocol.d.cts +10 -0
  360. package/dist/record/attachment/protocol.d.mts +10 -0
  361. package/dist/record/attachment/protocol.d.ts +10 -0
  362. package/dist/record/attachment/protocol.js +38 -36
  363. package/dist/record/codec/seal-manifest.d.cts +3 -3
  364. package/dist/record/codec/seal-manifest.d.mts +3 -3
  365. package/dist/record/codec/seal-manifest.d.ts +3 -3
  366. package/dist/record/family/adapter-usage/definition.d.cts +1 -0
  367. package/dist/record/family/adapter-usage/definition.d.mts +1 -0
  368. package/dist/record/family/adapter-usage/definition.d.ts +1 -0
  369. package/dist/record/family/adapter-usage/persistence.cjs +1 -1
  370. package/dist/record/family/adapter-usage/persistence.cjs.map +1 -1
  371. package/dist/record/family/adapter-usage/persistence.d.cts +2 -1
  372. package/dist/record/family/adapter-usage/persistence.d.mts +2 -1
  373. package/dist/record/family/adapter-usage/persistence.d.ts +2 -1
  374. package/dist/record/family/adapter-usage/schema.cjs +21 -2
  375. package/dist/record/family/adapter-usage/schema.cjs.map +1 -1
  376. package/dist/record/family/adapter-usage/schema.d.cts +85 -0
  377. package/dist/record/family/adapter-usage/schema.d.mts +85 -0
  378. package/dist/record/family/adapter-usage/schema.d.ts +85 -0
  379. package/dist/record/family/adapter-usage/schema.js +18 -12
  380. package/dist/record/family/catalog.cjs +31 -27
  381. package/dist/record/family/catalog.cjs.map +1 -1
  382. package/dist/record/family/catalog.d.cts +94 -3
  383. package/dist/record/family/catalog.d.mts +94 -3
  384. package/dist/record/family/catalog.d.ts +94 -3
  385. package/dist/record/family/current.cjs +45 -42
  386. package/dist/record/family/current.cjs.map +1 -1
  387. package/dist/record/family/current.d.cts +240 -1
  388. package/dist/record/family/current.d.mts +240 -1
  389. package/dist/record/family/current.d.ts +240 -1
  390. package/dist/record/family/execution-traces/definition.d.cts +26 -0
  391. package/dist/record/family/execution-traces/definition.d.mts +26 -0
  392. package/dist/record/family/execution-traces/definition.d.ts +26 -0
  393. package/dist/record/family/execution-traces/definition.js +20 -14
  394. package/dist/record/family/execution-traces/schema.cjs +57 -1
  395. package/dist/record/family/execution-traces/schema.cjs.map +1 -1
  396. package/dist/record/family/execution-traces/schema.d.cts +114 -0
  397. package/dist/record/family/execution-traces/schema.d.mts +114 -0
  398. package/dist/record/family/execution-traces/schema.d.ts +114 -0
  399. package/dist/record/family/execution-traces/schema.js +18 -12
  400. package/dist/record/family/judge-usage/definition.cjs +12 -0
  401. package/dist/record/family/judge-usage/definition.cjs.map +1 -0
  402. package/dist/record/family/judge-usage/definition.d.cts +91 -0
  403. package/dist/record/family/judge-usage/definition.d.mts +91 -0
  404. package/dist/record/family/judge-usage/definition.d.ts +91 -0
  405. package/dist/record/family/judge-usage/definition.js +6 -0
  406. package/dist/record/family/judge-usage/definition.js.map +1 -0
  407. package/dist/record/family/judge-usage/persistence.cjs +10 -0
  408. package/dist/record/family/judge-usage/persistence.cjs.map +1 -0
  409. package/dist/record/family/judge-usage/persistence.d.cts +91 -0
  410. package/dist/record/family/judge-usage/persistence.d.mts +91 -0
  411. package/dist/record/family/judge-usage/persistence.d.ts +91 -0
  412. package/dist/record/family/judge-usage/persistence.js +6 -0
  413. package/dist/record/family/judge-usage/persistence.js.map +1 -0
  414. package/dist/record/family/judge-usage/schema.cjs +307 -0
  415. package/dist/record/family/judge-usage/schema.cjs.map +1 -0
  416. package/dist/record/family/judge-usage/schema.d.cts +351 -0
  417. package/dist/record/family/judge-usage/schema.d.mts +351 -0
  418. package/dist/record/family/judge-usage/schema.d.ts +351 -0
  419. package/dist/record/family/judge-usage/schema.js +18 -0
  420. package/dist/record/family/judge-usage/schema.js.map +1 -0
  421. package/dist/record/host/sqlite-host.cjs +55 -9
  422. package/dist/record/host/sqlite-host.cjs.map +1 -1
  423. package/dist/record/host/types.cjs.map +1 -1
  424. package/dist/record/host/types.d.cts +7 -0
  425. package/dist/record/host/types.d.mts +7 -0
  426. package/dist/record/host/types.d.ts +7 -0
  427. package/dist/record/model/run-context.cjs +7 -1
  428. package/dist/record/model/run-context.cjs.map +1 -1
  429. package/dist/record/model/run-context.d.cts +4 -0
  430. package/dist/record/model/run-context.d.mts +4 -0
  431. package/dist/record/model/run-context.d.ts +4 -0
  432. package/dist/record/model/seal-manifest.d.cts +1 -1
  433. package/dist/record/model/seal-manifest.d.mts +1 -1
  434. package/dist/record/model/seal-manifest.d.ts +1 -1
  435. package/dist/record/sqlite/client.cjs +18 -5
  436. package/dist/record/sqlite/client.cjs.map +1 -1
  437. package/dist/record/sqlite/client.d.cts +1 -1
  438. package/dist/record/sqlite/client.d.mts +1 -1
  439. package/dist/record/sqlite/client.d.ts +1 -1
  440. package/dist/record/sqlite/project-state-database.cjs +29 -8
  441. package/dist/record/sqlite/project-state-database.cjs.map +1 -1
  442. package/dist/run/host/runtime.cjs +4 -0
  443. package/dist/run/host/runtime.cjs.map +1 -1
  444. package/dist/run/storage/sqlite.cjs +20 -0
  445. package/dist/run/storage/sqlite.cjs.map +1 -1
  446. package/dist/runner/adapter-execution-trace.cjs +125 -2
  447. package/dist/runner/adapter-execution-trace.cjs.map +1 -1
  448. package/dist/runner/adapter-execution-trace.d.cts +1 -1
  449. package/dist/runner/adapter-execution-trace.d.mts +1 -1
  450. package/dist/runner/adapter-execution-trace.d.ts +1 -1
  451. package/dist/runner/adapter-usage.cjs +55 -7
  452. package/dist/runner/adapter-usage.cjs.map +1 -1
  453. package/dist/runner/adapter-usage.d.cts +9 -2
  454. package/dist/runner/adapter-usage.d.mts +9 -2
  455. package/dist/runner/adapter-usage.d.ts +9 -2
  456. package/dist/runner/adoption.cjs +9 -0
  457. package/dist/runner/adoption.cjs.map +1 -1
  458. package/dist/runner/attempt-cancellation.cjs +12 -0
  459. package/dist/runner/attempt-cancellation.cjs.map +1 -0
  460. package/dist/runner/attempt-cancellation.d.cts +6 -0
  461. package/dist/runner/attempt-cancellation.d.mts +6 -0
  462. package/dist/runner/attempt-cancellation.d.ts +6 -0
  463. package/dist/runner/attempt-cancellation.js +6 -0
  464. package/dist/runner/attempt-cancellation.js.map +1 -0
  465. package/dist/runner/attempt-resources.cjs +10 -4
  466. package/dist/runner/attempt-resources.cjs.map +1 -1
  467. package/dist/runner/attempt-resources.d.cts +2 -1
  468. package/dist/runner/attempt-resources.d.mts +2 -1
  469. package/dist/runner/attempt-resources.d.ts +2 -1
  470. package/dist/runner/attempt.cjs +1177 -1139
  471. package/dist/runner/attempt.cjs.map +1 -1
  472. package/dist/runner/config-identity.cjs +20 -1
  473. package/dist/runner/config-identity.cjs.map +1 -1
  474. package/dist/runner/config-identity.d.cts +5 -0
  475. package/dist/runner/config-identity.d.mts +5 -0
  476. package/dist/runner/config-identity.d.ts +5 -0
  477. package/dist/runner/discover.cjs +14 -0
  478. package/dist/runner/discover.cjs.map +1 -1
  479. package/dist/runner/discover.d.cts +3 -0
  480. package/dist/runner/discover.d.mts +3 -0
  481. package/dist/runner/discover.d.ts +3 -0
  482. package/dist/runner/feedback/human.cjs +9 -4
  483. package/dist/runner/feedback/human.cjs.map +1 -1
  484. package/dist/runner/fingerprint.cjs +1 -0
  485. package/dist/runner/fingerprint.cjs.map +1 -1
  486. package/dist/runner/judge-usage.cjs +18 -0
  487. package/dist/runner/judge-usage.cjs.map +1 -0
  488. package/dist/runner/judge-usage.d.cts +5 -0
  489. package/dist/runner/judge-usage.d.mts +5 -0
  490. package/dist/runner/judge-usage.d.ts +5 -0
  491. package/dist/runner/judge-usage.js +8 -0
  492. package/dist/runner/judge-usage.js.map +1 -0
  493. package/dist/runner/project-target.cjs.map +1 -1
  494. package/dist/runner/project-target.d.cts +2 -0
  495. package/dist/runner/project-target.d.mts +2 -0
  496. package/dist/runner/project-target.d.ts +2 -0
  497. package/dist/runner/record/planning.cjs +1 -0
  498. package/dist/runner/record/planning.cjs.map +1 -1
  499. package/dist/runner/record.cjs +4 -0
  500. package/dist/runner/record.cjs.map +1 -1
  501. package/dist/runner/reporters/braintrust.cjs +2 -0
  502. package/dist/runner/reporters/braintrust.cjs.map +1 -1
  503. package/dist/runner/reuse-diagnostics.cjs +9 -10
  504. package/dist/runner/reuse-diagnostics.cjs.map +1 -1
  505. package/dist/runner/reuse-diagnostics.d.cts +5 -6
  506. package/dist/runner/reuse-diagnostics.d.mts +5 -6
  507. package/dist/runner/reuse-diagnostics.d.ts +5 -6
  508. package/dist/runner/reuse-diagnostics.js +4 -4
  509. package/dist/runner/reuse-plan.cjs +100 -48
  510. package/dist/runner/reuse-plan.cjs.map +1 -1
  511. package/dist/runner/reuse-plan.d.cts +60 -3
  512. package/dist/runner/reuse-plan.d.mts +60 -3
  513. package/dist/runner/reuse-plan.d.ts +60 -3
  514. package/dist/runner/reuse-plan.js +14 -10
  515. package/dist/runner/run.cjs +8 -6
  516. package/dist/runner/run.cjs.map +1 -1
  517. package/dist/runner/session.cjs +1 -1
  518. package/dist/runner/session.cjs.map +1 -1
  519. package/dist/runner/session.d.cts +1 -1
  520. package/dist/runner/session.d.mts +1 -1
  521. package/dist/runner/session.d.ts +1 -1
  522. package/dist/runner/source-producer.cjs +3 -1
  523. package/dist/runner/source-producer.cjs.map +1 -1
  524. package/dist/runner/types.cjs.map +1 -1
  525. package/dist/runner/types.d.cts +26 -5
  526. package/dist/runner/types.d.mts +26 -5
  527. package/dist/runner/types.d.ts +26 -5
  528. package/dist/sandbox/action.cjs +3 -0
  529. package/dist/sandbox/action.cjs.map +1 -1
  530. package/dist/sandbox/commands.cjs +3 -0
  531. package/dist/sandbox/commands.cjs.map +1 -1
  532. package/dist/sandbox/compose.cjs +6 -0
  533. package/dist/sandbox/compose.cjs.map +1 -1
  534. package/dist/sandbox/docker-profile/registry.cjs +7 -0
  535. package/dist/sandbox/docker-profile/registry.cjs.map +1 -1
  536. package/dist/sandbox/docker-profile/runtime.cjs +3 -0
  537. package/dist/sandbox/docker-profile/runtime.cjs.map +1 -1
  538. package/dist/sandbox/docker-profile/schema.cjs +7 -0
  539. package/dist/sandbox/docker-profile/schema.cjs.map +1 -1
  540. package/dist/sandbox/incus.cjs +7 -0
  541. package/dist/sandbox/incus.cjs.map +1 -1
  542. package/dist/sandbox/layer.cjs +12 -0
  543. package/dist/sandbox/layer.cjs.map +1 -1
  544. package/dist/shared/adapter-cleanup.cjs +18 -0
  545. package/dist/shared/adapter-cleanup.cjs.map +1 -0
  546. package/dist/shared/adapter-cleanup.d.cts +4 -0
  547. package/dist/shared/adapter-cleanup.d.mts +4 -0
  548. package/dist/shared/adapter-cleanup.d.ts +4 -0
  549. package/dist/shared/adapter-cleanup.js +10 -0
  550. package/dist/shared/adapter-cleanup.js.map +1 -0
  551. package/dist/shared/attempt-lifecycle.cjs +3 -0
  552. package/dist/shared/attempt-lifecycle.cjs.map +1 -0
  553. package/dist/shared/attempt-lifecycle.d.cts +13 -0
  554. package/dist/shared/attempt-lifecycle.d.mts +13 -0
  555. package/dist/shared/attempt-lifecycle.d.ts +13 -0
  556. package/dist/shared/attempt-lifecycle.js +4 -0
  557. package/dist/shared/attempt-lifecycle.js.map +1 -0
  558. package/dist/show/contribution.cjs +52 -14
  559. package/dist/show/contribution.cjs.map +1 -1
  560. package/dist/show/contribution.d.cts +23 -1
  561. package/dist/show/contribution.d.mts +23 -1
  562. package/dist/show/contribution.d.ts +23 -1
  563. package/dist/show/model.cjs +8 -1
  564. package/dist/show/model.cjs.map +1 -1
  565. package/dist/show/model.d.cts +10 -1
  566. package/dist/show/model.d.mts +10 -1
  567. package/dist/show/model.d.ts +10 -1
  568. package/dist/show/render.cjs +287 -49
  569. package/dist/show/render.cjs.map +1 -1
  570. package/dist/show/render.d.cts +3 -0
  571. package/dist/show/render.d.mts +3 -0
  572. package/dist/show/render.d.ts +3 -0
  573. package/dist/sources/projector.cjs +3 -0
  574. package/dist/sources/projector.cjs.map +1 -1
  575. package/dist/types.cjs.map +1 -1
  576. package/dist/types.d.cts +1 -1
  577. package/dist/types.d.mts +1 -1
  578. package/dist/types.d.ts +1 -1
  579. package/dist/view/app-dist/.vite/manifest.json +2 -2
  580. package/dist/view/app-dist/assets/index-CmVoXbwG.js +63 -0
  581. package/dist/view/app-dist/assets/{quality-cost-scatter-CFB3wnkN.js → quality-cost-scatter-BuLISaH4.js} +1 -1
  582. package/dist/view/app-dist/index.html +1 -1
  583. package/dist/view/cli/contribution.cjs +54 -21
  584. package/dist/view/cli/contribution.cjs.map +1 -1
  585. package/dist/view/cli/contribution.d.cts +2 -1
  586. package/dist/view/cli/contribution.d.mts +2 -1
  587. package/dist/view/cli/contribution.d.ts +2 -1
  588. package/dist/view/function-runtime.mjs +13231 -11647
  589. package/dist/view/http-protocol.cjs +3 -0
  590. package/dist/view/http-protocol.cjs.map +1 -1
  591. package/dist/view/http-protocol.d.cts +8 -0
  592. package/dist/view/http-protocol.d.mts +8 -0
  593. package/dist/view/http-protocol.d.ts +8 -0
  594. package/dist/view/inspection-host.cjs +17 -1
  595. package/dist/view/inspection-host.cjs.map +1 -1
  596. package/dist/view/render.cjs +1 -0
  597. package/dist/view/render.cjs.map +1 -1
  598. package/dist/view/render.d.cts +2 -1
  599. package/dist/view/render.d.mts +2 -1
  600. package/dist/view/render.d.ts +2 -1
  601. package/dist/view/revision.cjs.map +1 -1
  602. package/dist/view/revision.d.cts +12 -0
  603. package/dist/view/revision.d.mts +12 -0
  604. package/dist/view/revision.d.ts +12 -0
  605. package/dist/view/server.cjs +10 -9
  606. package/dist/view/server.cjs.map +1 -1
  607. package/docs-site/zh/examples/ai-agent-application.mdx +1 -1
  608. package/docs-site/zh/explanation/adapter.mdx +2 -2
  609. package/docs-site/zh/explanation/assert.mdx +60 -24
  610. package/docs-site/zh/explanation/drive.mdx +1 -1
  611. package/docs-site/zh/explanation/evals.mdx +31 -29
  612. package/docs-site/zh/explanation/experiment.mdx +15 -15
  613. package/docs-site/zh/explanation/judge.mdx +23 -15
  614. package/docs-site/zh/explanation/overview.mdx +44 -21
  615. package/docs-site/zh/explanation/record.mdx +28 -13
  616. package/docs-site/zh/explanation/results-flow.mdx +37 -0
  617. package/docs-site/zh/explanation/runner.mdx +27 -6
  618. package/docs-site/zh/index.mdx +22 -115
  619. package/docs-site/zh/introduction.mdx +89 -61
  620. package/docs-site/zh/reference/cli.mdx +50 -26
  621. package/docs-site/zh/reference/define-agent.mdx +37 -5
  622. package/docs-site/zh/reference/define-eval.mdx +130 -29
  623. package/docs-site/zh/reference/define-experiment.mdx +251 -0
  624. package/docs-site/zh/reference/events.mdx +34 -30
  625. package/docs-site/zh/reference/expect.mdx +77 -2
  626. package/docs-site/zh/reference/official-adapters.mdx +2 -2
  627. package/docs-site/zh/troubleshooting/debug-sandbox.mdx +43 -24
  628. package/docs-site/zh/troubleshooting/debugging.mdx +65 -42
  629. package/docs-site/zh/troubleshooting/recover-after-kill.mdx +35 -28
  630. package/docs-site/zh/tutorials/accept-results.mdx +34 -24
  631. package/docs-site/zh/tutorials/agent-feedback-loop.mdx +74 -43
  632. package/docs-site/zh/tutorials/agent-onboarding.mdx +27 -20
  633. package/docs-site/zh/tutorials/authoring.mdx +138 -121
  634. package/docs-site/zh/tutorials/ci-integration.mdx +46 -22
  635. package/docs-site/zh/tutorials/compare-runs.mdx +40 -13
  636. package/docs-site/zh/tutorials/concurrency.mdx +66 -62
  637. package/docs-site/zh/tutorials/configuration.mdx +17 -17
  638. package/docs-site/zh/tutorials/connect-otel.mdx +48 -37
  639. package/docs-site/zh/tutorials/connect-your-agent.mdx +80 -173
  640. package/docs-site/zh/tutorials/control-run-cost.mdx +63 -12
  641. package/docs-site/zh/tutorials/criteria-files.mdx +28 -21
  642. package/docs-site/zh/tutorials/custom-application.mdx +75 -55
  643. package/docs-site/zh/tutorials/dataset-fanout.mdx +32 -22
  644. package/docs-site/zh/tutorials/debug-lifecycle-plan.mdx +38 -31
  645. package/docs-site/zh/tutorials/docker-in-docker.mdx +83 -89
  646. package/docs-site/zh/tutorials/eval-groups.mdx +70 -89
  647. package/docs-site/zh/tutorials/evaluation-kinds.mdx +12 -5
  648. package/docs-site/zh/tutorials/experiment-services.mdx +243 -0
  649. package/docs-site/zh/tutorials/fixtures.mdx +42 -26
  650. package/docs-site/zh/tutorials/handle-execution-failures.mdx +41 -19
  651. package/docs-site/zh/tutorials/install-custom-sandbox-agent.mdx +21 -16
  652. package/docs-site/zh/tutorials/niceeval-taste-application.mdx +394 -0
  653. package/docs-site/zh/tutorials/niceeval-taste.mdx +190 -0
  654. package/docs-site/zh/tutorials/nixos-managed-dind.mdx +51 -39
  655. package/docs-site/zh/tutorials/plugins.mdx +55 -31
  656. package/docs-site/zh/tutorials/prebuilt-sandbox.mdx +204 -0
  657. package/docs-site/zh/tutorials/quickstart.mdx +91 -47
  658. package/docs-site/zh/tutorials/reporters.mdx +45 -30
  659. package/docs-site/zh/tutorials/rerun-and-cache.mdx +39 -40
  660. package/docs-site/zh/tutorials/sandbox-agent.mdx +65 -40
  661. package/docs-site/zh/tutorials/sandbox-providers.mdx +104 -282
  662. package/docs-site/zh/tutorials/sandbox-reuse.mdx +65 -97
  663. package/docs-site/zh/tutorials/select-and-preview.mdx +25 -20
  664. package/docs-site/zh/tutorials/verify-judge.mdx +49 -26
  665. package/docs-site/zh/tutorials/viewing-results.mdx +130 -80
  666. package/docs-site/zh/tutorials/write-experiment.mdx +116 -298
  667. package/docs-site/zh/tutorials/write-send.mdx +53 -52
  668. package/node_modules/undici/docs/docs/api/Agent.md +8 -8
  669. package/node_modules/undici/docs/docs/api/DiagnosticsChannel.md +9 -6
  670. package/node_modules/undici/docs/docs/api/Errors.md +25 -0
  671. package/node_modules/undici/docs/docs/api/Interceptors.md +4 -3
  672. package/node_modules/undici/docs/docs/api/Pool.md +5 -6
  673. package/node_modules/undici/lib/api/readable.js +1 -2
  674. package/node_modules/undici/lib/core/request.js +23 -2
  675. package/node_modules/undici/lib/core/util.js +0 -39
  676. package/node_modules/undici/lib/dispatcher/client-h1.js +11 -4
  677. package/node_modules/undici/lib/dispatcher/client-h2.js +52 -8
  678. package/node_modules/undici/lib/dispatcher/client.js +1 -1
  679. package/node_modules/undici/lib/dispatcher/dispatcher1-wrapper.js +3 -2
  680. package/node_modules/undici/lib/dispatcher/env-http-proxy-agent.js +21 -6
  681. package/node_modules/undici/lib/dispatcher/pool-base.js +63 -2
  682. package/node_modules/undici/lib/dispatcher/pool.js +90 -7
  683. package/node_modules/undici/lib/dispatcher/proxy-agent.js +16 -3
  684. package/node_modules/undici/lib/handler/deduplication-handler.js +24 -11
  685. package/node_modules/undici/lib/handler/redirect-handler.js +5 -0
  686. package/node_modules/undici/lib/handler/retry-handler.js +72 -13
  687. package/node_modules/undici/lib/interceptor/cache.js +4 -1
  688. package/node_modules/undici/lib/interceptor/decompress.js +236 -69
  689. package/node_modules/undici/lib/web/cache/cache.js +17 -8
  690. package/node_modules/undici/lib/web/cache/cachestorage.js +11 -5
  691. package/node_modules/undici/lib/web/cookies/index.js +19 -7
  692. package/node_modules/undici/lib/web/cookies/util.js +3 -1
  693. package/node_modules/undici/lib/web/eventsource/eventsource.js +25 -1
  694. package/node_modules/undici/lib/web/fetch/body.js +22 -15
  695. package/node_modules/undici/lib/web/fetch/data-url.js +6 -6
  696. package/node_modules/undici/lib/web/fetch/formdata.js +30 -36
  697. package/node_modules/undici/lib/web/fetch/headers.js +34 -31
  698. package/node_modules/undici/lib/web/fetch/index.js +38 -14
  699. package/node_modules/undici/lib/web/fetch/request.js +74 -79
  700. package/node_modules/undici/lib/web/fetch/response.js +43 -44
  701. package/node_modules/undici/lib/web/fetch/util.js +7 -7
  702. package/node_modules/undici/lib/web/infra/index.js +16 -1
  703. package/node_modules/undici/lib/web/webidl/index.js +2 -14
  704. package/node_modules/undici/lib/web/websocket/events.js +43 -26
  705. package/node_modules/undici/lib/web/websocket/receiver.js +5 -0
  706. package/node_modules/undici/lib/web/websocket/stream/websocketerror.js +13 -10
  707. package/node_modules/undici/lib/web/websocket/websocket.js +44 -41
  708. package/node_modules/undici/package.json +1 -1
  709. package/node_modules/undici/types/diagnostics-channel.d.ts +2 -1
  710. package/node_modules/undici/types/interceptors.d.ts +1 -1
  711. package/node_modules/undici/types/webidl.d.ts +13 -7
  712. package/package.json +4 -4
  713. package/dist/view/app-dist/assets/index-DFzw1L97.js +0 -63
  714. package/docs-site/zh/tutorials/experiments.mdx +0 -101
@@ -1,48 +1,69 @@
1
1
  ---
2
- title: "为你的 Agent 项目设置评估"
2
+ title: "快速开始:跑通第一条评估用例"
3
3
  sidebarTitle: "快速开始"
4
- description: "安装 NiceEval,写三个文件,10 分钟内对你自己的应用跑通第一条评估用例。"
4
+ description: "安装 NiceEval,写一个 Adapter、一个 Experiment 和一条评估用例,十分钟左右对你自己的应用跑通第一条评估,并在终端和浏览器里看到结果。"
5
5
  ---
6
6
 
7
- 接入应用需要三个文件:一个 **Adapter** 负责调用应用,一个 **Experiment** 固定被测对象和运行次数,一条**评估用例**定义断言。以下步骤提供完整的最短路径,并分别覆盖 Coding Agent 自动接入和人工接入。
7
+ 这一页带你对自己的应用跑通第一条评估用例。你会写三个文件:
8
+
9
+ - **Adapter**:告诉 NiceEval 怎样调用你的应用。
10
+ - **Experiment**:说明对着哪个 Adapter、用哪个模型跑。
11
+ - **评估用例**:说明发什么消息、怎样算做对。
12
+
13
+ 有两种走法。让 Coding Agent 帮你写最快;自己动手写能更清楚每个文件管什么。
14
+
15
+ ## 让 Coding Agent 帮你接入
8
16
 
9
- ## 使用 Coding Agent 接入(推荐)
10
17
  <Steps>
11
- <Step title="接入">
12
- 把以下指令发送给你正在使用的 Coding Agent(例如 Codex、Claude Code 或其他能修改当前项目的 Agent):
18
+ <Step title="把指令发给 Agent">
19
+ 在当前项目里打开你正在用的 Coding Agent,例如 Claude Code 或 Codex,发送下面这段话:
13
20
 
14
21
  ```text
15
22
  阅读 https://niceeval.com/INIT.md,为当前仓库安装并接入 niceeval,端到端跑通第一条评估用例。完成时列出创建的 Experiment ID、运行命令和结果查看命令。
16
-
17
23
  ```
24
+
25
+ Agent 会先探索你的项目,确认要测哪个接口,再写出三个文件并跑一次。
18
26
  </Step>
19
- <Step title="运行测试">
20
- 使用 Agent 最后列出的 Experiment ID。若它没有列出,先运行 `pnpm exec niceeval exp list` 查看可用的 Experiment。
27
+ <Step title="自己再跑一次">
28
+ 用 Agent 最后列出的 Experiment ID 运行。它没有列出时,先用 `pnpm exec niceeval exp list` 查看有哪些 Experiment:
21
29
 
22
30
  ```bash
23
31
  pnpm exec niceeval exp <experiment-id>
24
32
  ```
25
33
  </Step>
26
- <Step title="查看结果">
34
+ <Step title="看结果">
27
35
  ```bash
28
- pnpm exec niceeval query discover # AI、脚本或 CI 先发现读取 operation
29
- pnpm exec niceeval view # 网页审阅已发布的结果
36
+ pnpm exec niceeval show
30
37
  ```
38
+
39
+ 终端里会列出每条评估用例的结果,失败的排在前面。结果怎么读,见下文[读懂结果](#读懂结果)。
31
40
  </Step>
32
41
  </Steps>
33
42
 
43
+ ## 自己动手接入
44
+
45
+ ### 准备被测应用
34
46
 
35
- ## 人工接入
47
+ 先把你的应用启动起来,并确认它有一个能调用的接口。下面的示例假设:
36
48
 
37
- 先启动你的被测应用,并确认它有一个可调用接口。下方示例假设接口是 `POST http://localhost:3000/chat`,请求体接收 `message` 和 `model`,响应 JSON 含有字符串字段 `reply`。接口不同就改 Adapter 的 URL、请求体和读取响应的代码;还没有可启动的应用时,先从 [Examples](/zh/examples) 选择一个可运行项目。
49
+ - 接口是 `POST http://localhost:3000/chat`;
50
+ - 请求体是 `{ "message": "...", "model": "..." }`;
51
+ - 响应体是 `{ "reply": "..." }`。
52
+
53
+ 你的接口不一样也没关系,只需要改 Adapter 里的地址、请求体和读取回复的那一行。手边还没有能启动的应用时,先从 [Examples](/zh/examples) 挑一个可运行项目。
54
+
55
+ ### 安装
38
56
 
39
57
  ```bash
40
58
  pnpm add -D niceeval
41
59
  pnpm exec niceeval init
42
60
  ```
43
61
 
44
- ### Adapter
45
- Adapter 连接 Agent 与 [NiceEval](https://niceeval.com/)。三种接入等级从无侵入连接到修改 Agent 配置,分别提供不同的评估能力,见 [Tier](/zh/explanation/tier)。
62
+ `init` 会创建 `niceeval.config.ts`。它还会在 `AGENTS.md` 或 `CLAUDE.md` 里写一段说明,让你的 Coding Agent 读取和安装版本匹配的文档。
63
+
64
+ ### 写 Adapter
65
+
66
+ Adapter 负责一件事:把评估用例里的消息发给你的应用,再把回复交回给 NiceEval。
46
67
 
47
68
  ```ts
48
69
  // agents/my-agent.ts
@@ -52,11 +73,11 @@ export default defineAgent({
52
73
  name: "my-agent",
53
74
  evidenceCoverage: completeEvidenceCoverage,
54
75
  async send(input, ctx) {
55
- // 示例地址:换成你自己 agent 的真实端点(HTTP、CLI、SDK 都行,只要 send 里能拿到回复文本)
76
+ // 换成你自己应用的地址
56
77
  const r = await fetch("http://localhost:3000/chat", {
57
78
  method: "POST",
58
79
  headers: { "content-type": "application/json" },
59
- body: JSON.stringify({ message: input.text, model: ctx.model }), // ← experiment.model 经 ctx.model 到这里
80
+ body: JSON.stringify({ message: input.text, model: ctx.model }), // ctx.model 来自 Experiment
60
81
  signal: ctx.signal,
61
82
  });
62
83
  const body = await r.json();
@@ -68,7 +89,9 @@ export default defineAgent({
68
89
  });
69
90
  ```
70
91
 
71
- ### Experiment
92
+ `events` 是 NiceEval 读懂回复的方式。这里只交回一条助手消息,后面可以再加工具调用、用量等事件。
93
+
94
+ ### 写 Experiment
72
95
 
73
96
  ```ts
74
97
  // experiments/my-agent.ts
@@ -77,13 +100,15 @@ import myAgent from "../agents/my-agent.ts";
77
100
 
78
101
  export default defineExperiment({
79
102
  description: "my-agent 基线",
80
- model: "gpt-4o",
81
103
  agent: myAgent,
104
+ model: "gpt-4o",
82
105
  attempts: 1,
83
106
  });
84
107
  ```
85
108
 
86
- ### 评估用例
109
+ 文件名就是 Experiment ID,这里是 `my-agent`。以后想对比另一个模型,复制一份改 `model` 就行,评估用例不用动。
110
+
111
+ ### 写评估用例
87
112
 
88
113
  ```ts
89
114
  // evals/refund-policy.eval.ts
@@ -91,44 +116,63 @@ import { defineEval } from "niceeval";
91
116
  import { pattern } from "niceeval/expect";
92
117
 
93
118
  export default defineEval({
94
- description: "退款政策问答",
119
+ description: "能正确回答退款政策",
95
120
  async test(t) {
96
- await t.send("你们的退款政策是什么?");
121
+ await t.send("你们的退款政策是什么?");
97
122
  t.succeeded();
98
- t.check(t.reply, pattern(/30\s*天|一个月/)); // 措辞会变,用等价正则而不是单一精确短语
123
+ t.check(t.reply, pattern(/30\s*天|一个月/));
99
124
  },
100
125
  });
101
126
  ```
102
127
 
128
+ 模型每次的措辞都会变,所以这里用正则匹配“30 天”或“一个月”,而不是要求一字不差。
129
+
130
+ ### 运行
131
+
103
132
  ```bash
104
- pnpm exec niceeval exp my-agent # 跑起来
105
- pnpm exec niceeval query discover # 让机器发现可用 operation
106
- pnpm exec niceeval view # 在网页审阅当前已发布的结果
133
+ pnpm exec niceeval exp my-agent
107
134
  ```
108
135
 
109
- 到这里第一条评估用例已经跑通。
136
+ 运行过程中终端会显示进度。结束后你会看到类似这样的输出:
110
137
 
111
- 这个最小 Adapter 只支持**单轮**调用,第二轮不会携带第一轮的历史。多轮会话、工具调用事件(供 `t.calledTool()` 使用)、HITL 和 tracing 都是后续可选能力。按[接入你的 Agent](/zh/tutorials/connect-your-agent)继续扩展时,已有评估用例不需要修改。
138
+ ```text
139
+ ╭─ PASSED ─────────────────────────────────────────────────────── 12s ─╮
140
+ │ 1 passed · 0 failed · 0 errored (0 reused) │
141
+ ╰──────────────────────────────────────────────────────────────────────╯
112
142
 
113
- ## 对照完整项目
143
+ ╭─ RESULTS ───────────────────────────────────── 1 run configuration ─╮
144
+ │ my-agent │
145
+ │ refund-policy 1/1 passed │
146
+ ╰──────────────────────────────────────────────────────────────────────╯
114
147
 
115
- 第一条评估用例跑通后,再到 [Examples](/zh/examples) 按被测对象选择完整项目。那里提供可运行源码。通用操作步骤在 Tutorials 的对应任务页中。
148
+ ╭─ NEXT ───────────────────────────────────────────────────────────────╮
149
+ │ my-agent │
150
+ │ show: niceeval show --run 8f3d6f62-1d34-4cf3-99c7-84ba3c483706 │
151
+ │ view: niceeval view --run 8f3d6f62-1d34-4cf3-99c7-84ba3c483706 │
152
+ ╰──────────────────────────────────────────────────────────────────────╯
153
+ ```
116
154
 
117
- ## 放进 CI
155
+ 到这里,第一条评估用例已经跑通。
118
156
 
119
- ```yaml
120
- name: evals
121
- on: [pull_request]
122
- jobs:
123
- evals:
124
- runs-on: ubuntu-latest
125
- steps:
126
- - uses: actions/checkout@v4
127
- - uses: actions/setup-node@v4
128
- - run: npm ci
129
- - run: npm exec niceeval exp my-agent
130
- ```
157
+ ## 读懂结果
158
+
159
+ 标题告诉你这次运行的结论:
160
+
161
+ | 标题 | 意思 | 先做什么 |
162
+ |---|---|---|
163
+ | `PASSED` | 所有断言都满足了。 | 继续加评估用例。 |
164
+ | `FAILED` | 应用正常回复了,但至少一条断言没满足。 | 看 Agent 实际回了什么,判断是应用的问题还是断言写得太严。 |
165
+ | `ERRORED` | 没能拿到结果,例如应用没启动、接口报错、Adapter 抛了异常。 | 看 `FAILURES` 面板里的 `error:` 行,通常能直接看出原因;需要更多细节时运行它下面的 `details:` 命令。 |
166
+
167
+ 失败的 Attempt 下面都有一行 `details: niceeval show @<locator>`,复制它就能在终端里看到这一次运行的对话和断言。看整次运行,复制 `NEXT` 面板里的 `show:` 命令;想在浏览器里翻对话、看耗时,复制 `view:` 那一行。
168
+
169
+ 更多查看方式见[查看运行结果](/zh/tutorials/viewing-results)。
170
+
171
+ ## 接下来
172
+
173
+ 这个最小 Adapter 只支持单轮对话,第二轮不会带上第一轮的历史,也看不到工具调用。按你的下一步需要选:
131
174
 
132
- <Tip>
133
- 阅读 [编写评估用例](/zh/tutorials/authoring) 和 [题型与断言](/zh/tutorials/evaluation-kinds),再把示例替换成真实场景。
134
- </Tip>
175
+ - 要测多轮对话、工具调用或用量:按[编写 Send](/zh/tutorials/write-send)扩展 Adapter,已有评估用例不用改。
176
+ - 要写更多、更有区分度的评估用例:读[编写评估用例](/zh/tutorials/authoring)。
177
+ - 要比较两个模型或两版 prompt:读[编写实验](/zh/tutorials/write-experiment)。
178
+ - 要在 PR 上自动跑:读 [CI 集成](/zh/tutorials/ci-integration)。
@@ -1,63 +1,78 @@
1
1
  ---
2
- title: "将当前运行接入外部系统"
2
+ title: "把运行结果上报给 Braintrust 等外部系统"
3
3
  sidebarTitle: "Reporter 上报"
4
- description: "使用 Reporter、JUnit 和 receipt 把当前 Invocation 的信息交给外部系统,同时保持 Record 的所有权边界。"
4
+ description: "团队已经在用 Braintrust 或自己的看板时,用 Reporter 在运行过程中把结果发过去;CI 平台用 JUnit;后续脚本用 receipt 里的 Run ID 读取结果。"
5
5
  ---
6
6
 
7
- Reporter 面向当前 Invocation。它可以把当前运行的结果发送到外部观察或分析系统,但它不拥有已发布结果,也不能替代之后的固定 `query` 或 `view`。
7
+ 你的团队已经在 Braintrust 上看实验结果,或者有一个内部看板。你希望 NiceEval 跑完后,结果自动出现在那里,不用手动导出。
8
+
9
+ Reporter 做的就是这件事:在这一次运行过程中,把结果发给外部系统。它只负责“这一次”的上报,NiceEval 保存的结果不会因此改变。事后要重新读取、比较或分享结果,仍然用 `show`、`view` 和 `query`。
10
+
11
+ 一次 `niceeval exp` 会同时产出这几样东西:
8
12
 
9
13
  ```text
10
- 当前 Invocation
11
- ├─ 终端与 --json 反馈
14
+ 这一次运行
15
+ ├─ 终端输出,或 --json 的 NDJSON
12
16
  ├─ Reporter → 外部系统
13
- ├─ --junit → CI 平台文件
14
- └─ receipt → invocationId 与 runIds
17
+ ├─ --junit → 给 CI 平台的 JUnit 文件
18
+ └─ receipt → invocationId 与 createdRunIds
15
19
 
16
- 已发布结果
17
- └─ Run、Member、Attempt 与可查看的运行事实
20
+ 保存下来的结果(.niceeval/record.sqlite)
21
+ └─ Run、Attempt 和可以查看的运行记录
18
22
  ```
19
23
 
20
- ## Reporter 适合什么
24
+ ## 选择合适的上报方式
25
+
26
+ | 你要 | 用 |
27
+ | --- | --- |
28
+ | 把结果推到 Braintrust 或内部看板 | Reporter |
29
+ | 让 CI 平台按测试用例显示结果 | `--junit <path>` |
30
+ | 让后续脚本找到这次运行的结果 | `--json` 输出的 receipt |
31
+
32
+ ## 用 Reporter 推送到外部系统
21
33
 
22
- 把 Reporter 配在需要观察的运行范围,让它处理当前进程可见的结果。外部系统可以接收运行摘要、Attempt 结果或链接,但要读取完整业务事实时,应使用 receipt 的 `runIds` 重新打开已发布事实。
34
+ 在 `niceeval.config.ts` 的 `reporters` 里配置项目默认的 Reporter,评估用例的 `reporters` 会和它合并。字段说明见 [defineConfig 参考](/zh/reference/define-config)。
35
+
36
+ Reporter 能拿到的是这次运行过程中已知的结果。外部系统可以接收运行摘要、每个 Attempt 的结果或查看链接。要读取完整的结果,用 receipt 里的 Run ID 重新打开保存下来的结果。
23
37
 
24
38
  Reporter 不能:
25
39
 
26
- - 把外部系统的数据当作 Record 的业务值;
27
- - 用外部上报替代 Verdict、用量、断言或诊断;
28
- - 让后续固定 `query` 或 `view` 从外部系统取数;
29
- - 从外部系统把结果写回 Record。
40
+ - 把外部系统里的数据当作 NiceEval 的结果;
41
+ - 代替 NiceEval 保存的判定、用量、断言或诊断;
42
+ - 让 `query`、`show` 或 `view` 改从外部系统读数据;
43
+ - 把外部系统的结果写回 `.niceeval/record.sqlite`。
30
44
 
31
- ## 在 CI 中使用 JUnit
45
+ ## 在 CI 中输出 JUnit
32
46
 
33
- JUnit 是最直接的外部集成方式:
47
+ 接入 CI 平台最直接的方式是 JUnit:
34
48
 
35
49
  ```sh
36
- npx niceeval exp ci --junit ./artifacts/niceeval-junit.xml
50
+ pnpm exec niceeval exp ci --junit ./artifacts/niceeval-junit.xml
37
51
  ```
38
52
 
39
- NiceEval 以原子方式写出该文件,CI 平台读取它来显示注解和测试结果。它只描述这次调用的交付格式,不会成为 Record 的事实 owner。
53
+ NiceEval 先写临时文件,再一次性替换目标文件,CI 平台不会读到写了一半的内容。JUnit 只描述这一次运行,结果仍以 NiceEval 保存的为准。完整的 CI 配置见[在 CI 里自动跑评估](/zh/tutorials/ci-integration)。
40
54
 
41
- ## 使用 JSON 与 receipt
55
+ ## 用 receipt 关联后续步骤
42
56
 
43
57
  ```sh
44
- npx niceeval exp checkout --json
58
+ pnpm exec niceeval exp checkout --json
45
59
  ```
46
60
 
47
- `--json` 输出当前进程的 NDJSON。最后一条 receipt 包含 `invocationId`、`runIds`、开始时间、结束时间和 completion。外部自动化可以保留 receipt,再根据 `runIds` 选择要查看的固定 Inspection result 或 View。
61
+ `--json` 输出这次运行的 NDJSON。最后一行是 receipt,包含 `invocationId`、`createdRunIds`、`publicationCutoff`、开始时间、结束时间和 `completion`。外部自动化可以保存 receipt,再用 `createdRunIds` 打开对应的 Run:
48
62
 
49
63
  ```sh
64
+ pnpm exec niceeval show --run 01J9ZK3M6P4T7V9X2C5N8QW0RY
50
65
  pnpm exec niceeval view --run 01J9ZK3M6P4T7V9X2C5N8QW0RY
51
66
  ```
52
67
 
53
- 不要把 progress 或 diagnostic 行当作长期结果格式。它们服务当前 Invocation,完整业务事实属于已发布 Record。
68
+ 脚本需要结构化结果时,先运行 `pnpm --silent exec niceeval query discover` 找到可用的查询。
54
69
 
55
- ## 接入外部系统时的原则
70
+ 不要把进度或诊断行当作长期保存的结果格式。它们只描述这一次运行正在发生什么。
56
71
 
57
- 1. 让外部上报只消费当前 Invocation 已知的信息。
58
- 2. 将凭据保留在运行变量中,不写入输出内容。
59
- 3. 用退出状态处理 CI 门禁。
60
- 4. 用 receipt 的 `runIds` 把外部工作流关联到可查看的结果。
61
- 5. 需要审阅已发布事实时,使用 receipt 的 `runIds` 打开 `view`,或用 `query` 交给自动化流程。
72
+ ## 接入外部系统时的原则
62
73
 
63
- 有关 CI 命令与 JUnit,请阅读[CI 集成](/zh/tutorials/ci-integration)。
74
+ 1. 外部上报只使用这次运行中已知的信息。
75
+ 2. 凭据放在环境变量里,不写进任何输出。
76
+ 3. CI 门禁看退出状态。
77
+ 4. 用 receipt 的 `createdRunIds` 把外部工作流和 NiceEval 保存的结果关联起来。
78
+ 5. 需要审阅结果时,人用 `show` 或 `view` 打开那个 Run,自动化流程用 `query`。
@@ -1,81 +1,80 @@
1
1
  ---
2
- title: "重跑与沿用结果"
2
+ title: "修改后只重跑受影响的评估"
3
3
  sidebarTitle: "重跑与沿用"
4
- description: "查看每个 slot 会执行、自动采用还是由操作者接受,并理解这些关系怎样进入 Record。"
4
+ description: "改了 prompt、评估用例或模型之后,用 --dry 看清哪些结果会沿用、哪些会重跑,再用 --rerun 和 accept 控制范围。"
5
5
  ---
6
6
 
7
- 每次 Invocation 都建立新的 Run。Run 的 expected slots 定义本次分母;每个 slot 最多有一个 Member。Member 表示 slot 被哪个 Attempt 完整占据:
7
+ 你修好了 Agent 的一个工具调用 bug,想确认它真的修好了。一整套评估要跑一个小时、花几十美元,而这次改动只影响其中几道题。
8
8
 
9
- | 关系 | 含义 |
10
- | --- | --- |
11
- | `origin` | Attempt 的实际执行 slot 就是当前 slot。 |
12
- | `reference` | 采用了其它 Run 中已发布的 Attempt。 |
9
+ NiceEval 会记住每道题上次的结果。再次运行同一个 Experiment 时,没受这次改动影响、上次已经有结论的题直接沿用旧结果,其余的重新跑。你不需要手动挑题。
13
10
 
14
- 每个位置怎样形成(执行、自动沿用或操作者接受)都会随目标 Run 保存。Attempt 始终保留实际执行它的 `origin`;采用不会复制 Verdict、用量或其他业务值。
11
+ ## 运行前先看计划
15
12
 
16
- ## 先查看计划
17
-
18
- 在运行前使用 `--dry`。它根据已发布结果逐个列出执行或采用的原因。
13
+ 加上 `--dry`,NiceEval 只列计划,不调用模型,也不创建 Sandbox:
19
14
 
20
15
  ```sh
21
- npx niceeval exp checkout --dry
16
+ pnpm exec niceeval exp checkout --dry
22
17
  ```
23
18
 
24
- 示例输出:
25
-
26
19
  ```text
27
20
  PLAN
28
21
  checkout cart/add-item ordinal 0 reuse @1K1P0VJAPVJ12
29
22
  checkout cart/add-item ordinal 1 execute: input-identity-changed
30
23
  ```
31
24
 
32
- 如果信息缺失、无效、不支持或 domain 不同,规划器会说明原因并执行该 slot,而不会把它当作可采用结果。
25
+ 每一行是一次 Attempt。`reuse` 后面是要沿用的旧结果 locator;`execute` 后面是要重跑的原因。上面第二行的 `input-identity-changed` 表示这道题的输入变了,例如你改了评估用例、Fixture 或模型。
33
26
 
34
- ## 自动采用的条件
27
+ 结果和你预期的不一样时,先看原因。例如你只改了报告里的分组名,却看到整批 `execute`,说明某个会影响结果的输入也一起变了。
35
28
 
36
- 一个 Attempt 要能被自动采用,需要同时满足:
29
+ ## 哪些结果会被沿用
37
30
 
38
- - Verdict 是可采用的终态;
39
- - 输入身份与当前输入相同;
40
- - 执行时长符合当前 timeout;
41
- - 本次 `--rerun` 允许采用该 Verdict;
42
- - 本次没有通过 `--keep-sandbox` 要求真实执行。
31
+ 一个旧 Attempt 同时满足下面几条才会被自动沿用:
43
32
 
44
- 输入身份和配置身份都是带 domain 的不透明值。只有 domain 逐字相同才比较 value。Verdict 与资格信息为规划提供长期依据。
33
+ - 它已经有明确结论,`passed` 或 `failed`。`errored` 说明上次没跑成,总会重跑。
34
+ - 评估用例、Fixture、模型、Agent、判分标准和运行条件都和这次相同。
35
+ - 它的耗时没有超过这次的 `timeout`。
36
+ - 这次的 `--rerun` 没有要求重跑它。
37
+ - 这次没有用 `--keep-sandbox` 要求真实执行。
45
38
 
46
- 自动采用时,资格、身份和理由会随目标 Run 保存。Member 只保留引用,不复制这些说明,也不复制 Attempt 的业务值。
39
+ 只改 Experiment 的 `description` 或 `labels` 不会让旧结果失效。评估用例文件不同:文件内容一变,这道题就会重跑。
47
40
 
48
41
  ## 控制重跑范围
49
42
 
50
43
  ```sh
51
- # 默认采用符合条件的 passed 与 failed Attempt
52
- npx niceeval exp checkout
44
+ # 默认:沿用符合条件的 passed 和 failed
45
+ pnpm exec niceeval exp checkout
53
46
 
54
- # 单独写 --rerun 等价于 --rerun failed;已经 passed 的 slot 仍会采用
55
- npx niceeval exp checkout --rerun
47
+ # 修复 bug 后复验:重跑上次 failed 的,passed 的继续沿用
48
+ pnpm exec niceeval exp checkout --rerun
56
49
 
57
- # 重新执行选中范围内的全部 slot
58
- npx niceeval exp checkout --rerun all
50
+ # 模型供应商更新了服务、本地文件却没变:全部重跑
51
+ pnpm exec niceeval exp checkout --rerun all
59
52
  ```
60
53
 
61
- 改变 `--rerun` 只影响本次 Invocation,不会修改已有 Run 或 Attempt。
54
+ 单独写 `--rerun` 等于 `--rerun failed`,已经通过的题不会重跑。要连通过的一起重做,必须写 `--rerun all`。
62
55
 
63
- 不要在 `exp` 仍运行时并发执行 `show --experiment`:Experiment 可能尚未产生任何已发布结果。等待命令完成,从 receipt 复制本次 `runIds`,再用 `niceeval show --run <run-id>` 审阅确定的 Run。若目的是重做所有已通过或已评分的 slot,必须明确写 `--rerun all`;不带值的 `--rerun` 不会做到这一点。
56
+ `--rerun` 只影响这一次运行,不会改动已有的结果。
64
57
 
65
- ## 明确接受已有 Attempt
58
+ ## 确认旧结果在新配置下仍然成立
66
59
 
67
- 身份变化不一定表示历史结果已经失效,但 NiceEval 不会替操作者猜测。先从 dry run 或结果页取得完整 locator,核对候选 Attempt;确认它在新配置下仍然成立时,明确接受它:
60
+ 有时改动会让 NiceEval 判定旧结果不能沿用,但你知道它其实不受影响。例如你给 Agent 加了一个日志开关,这道题的行为完全没变。NiceEval 不会替你猜,需要你明确接受:
68
61
 
69
62
  ```sh
70
- npx niceeval accept @1K1P0VJAPVJ12
63
+ pnpm exec niceeval accept @1K1P0VJAPVJ12
71
64
  ```
72
65
 
73
- `accept` 会发布一个新的采用 Run,不会修改原 Attempt。一次需要接受多个结果时,把全部 locator 放在同一条命令中;NiceEval 会先原子预检所有输入,避免只接受一半。
66
+ `accept` 会新建一个 Run 引用这个旧 Attempt,原 Attempt 保持不变。要接受多条时,把所有 locator 写进同一条命令,NiceEval 会先检查全部输入,不会只接受一半。
67
+
68
+ 怎样取得候选 locator、接受前要核对什么,见[审阅并采用历史 Attempt](/zh/tutorials/accept-results)。
74
69
 
75
- 候选 locator 的取得方式、审阅项目、原子预检和失败处理见[审阅并采用历史 Attempt](/zh/tutorials/accept-results)。
70
+ ## 跑完之后看结果
76
71
 
77
- ## 发布后不可修改
72
+ 运行结束时,终端的 `NEXT` 面板会列出这次新建的 Run,并给出在终端和浏览器里查看它的两条命令:
73
+
74
+ ```sh
75
+ pnpm exec niceeval show --run <run-id>
76
+ ```
78
77
 
79
- Attempt 发布后不可修改。每一次读取都使用当时 Record 中已发布的值;新 Run 或新事实只会以新的发布出现。若 Attempt 或引用无效,固定 operation 将该位置标为 `core-invalid`,而不是 `not-recorded`。
78
+ 沿用的题和重跑的题都会出现在这个 Run 里,通过率按完整的一批题计算。不要在 `exp` 还没结束时用 `show --experiment` 看结果,那时可能还没有任何已发布的结果。
80
79
 
81
- 完成运行后,用 receipt 的 `runIds` 选择结果。查看步骤见[查看结果](/zh/tutorials/viewing-results)。
80
+ 查看结果的完整方式见[查看运行结果](/zh/tutorials/viewing-results)。沿用和重跑的判定原理见 [Runner](/zh/explanation/runner)。