niceeval 0.9.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/INDEX.md +31 -33
- package/dist/agents/types.d.ts +20 -17
- package/dist/context/types.d.ts +21 -21
- package/dist/i18n/en.d.ts +10 -2
- package/dist/i18n/zh-CN.d.ts +15 -7
- package/dist/o11y/execution-tree.d.ts +103 -0
- package/dist/o11y/otlp/select.d.ts +22 -0
- package/dist/report/{react → assets}/colors.d.ts +8 -0
- package/dist/report/{react → assets}/colors.js +23 -0
- package/dist/report/built-in/index.d.ts +2 -2
- package/dist/report/built-in/index.js +2 -2
- package/dist/report/built-in/standard.d.ts +8 -1
- package/dist/report/built-in/standard.js +17 -7
- package/dist/report/components/attempt-detail/AttemptAssertions.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptAssertions.js +17 -0
- package/dist/report/components/attempt-detail/AttemptConversation.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptConversation.js +34 -0
- package/dist/report/components/attempt-detail/AttemptDiagnostics.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptDiagnostics.js +7 -0
- package/dist/report/components/attempt-detail/AttemptDiff.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptDiff.js +11 -0
- package/dist/report/components/attempt-detail/AttemptError.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptError.js +16 -0
- package/dist/report/components/attempt-detail/AttemptFixPrompt.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptFixPrompt.js +7 -0
- package/dist/report/components/attempt-detail/AttemptSource.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptSource.js +27 -0
- package/dist/report/components/attempt-detail/AttemptSummary.d.ts +8 -0
- package/dist/report/components/attempt-detail/AttemptSummary.js +20 -0
- package/dist/report/components/attempt-detail/AttemptTimeline.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptTimeline.js +28 -0
- package/dist/report/components/attempt-detail/AttemptTrace.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptTrace.js +29 -0
- package/dist/report/components/attempt-detail/AttemptUsage.d.ts +6 -0
- package/dist/report/components/attempt-detail/AttemptUsage.js +20 -0
- package/dist/report/components/attempt-detail/compute.d.ts +24 -0
- package/dist/report/components/attempt-detail/compute.js +254 -0
- package/dist/report/components/attempt-detail/faces.d.ts +14 -0
- package/dist/report/components/attempt-detail/faces.js +235 -0
- package/dist/report/components/attempt-detail/index.d.ts +38 -0
- package/dist/report/components/attempt-detail/index.js +527 -0
- package/dist/report/{react → components}/cell.d.ts +5 -3
- package/dist/report/{react → components}/cell.js +3 -3
- package/dist/report/{react → components/entity-lists}/AttemptList.d.ts +8 -4
- package/dist/report/{react → components/entity-lists}/AttemptList.js +16 -10
- package/dist/report/{react → components/entity-lists}/EvalList.d.ts +3 -3
- package/dist/report/{react → components/entity-lists}/EvalList.js +0 -0
- package/dist/report/{react → components/entity-lists}/ExperimentList.d.ts +3 -3
- package/dist/report/{react → components/entity-lists}/ExperimentList.js +10 -10
- package/dist/report/components/entity-lists/compute.d.ts +23 -0
- package/dist/report/components/entity-lists/compute.js +171 -0
- package/dist/report/components/entity-lists/faces.d.ts +5 -0
- package/dist/report/components/entity-lists/faces.js +174 -0
- package/dist/report/components/entity-lists/index.d.ts +51 -0
- package/dist/report/components/entity-lists/index.js +165 -0
- package/dist/report/{react → components}/fixtures.d.ts +2 -2
- package/dist/report/{react → components}/fixtures.js +1 -1
- package/dist/report/{react → components/metric-views}/DeltaTable.d.ts +2 -2
- package/dist/report/{react → components/metric-views}/DeltaTable.js +4 -4
- package/dist/report/{react → components/metric-views}/MetricBars.d.ts +3 -3
- package/dist/report/{react → components/metric-views}/MetricBars.js +3 -3
- package/dist/report/{react → components/metric-views}/MetricLine.d.ts +2 -2
- package/dist/report/{react → components/metric-views}/MetricLine.js +8 -5
- package/dist/report/{react → components/metric-views}/MetricMatrix.d.ts +3 -3
- package/dist/report/{react → components/metric-views}/MetricMatrix.js +4 -4
- package/dist/report/components/metric-views/MetricScatter.d.ts +11 -0
- package/dist/report/{react → components/metric-views}/MetricScatter.js +26 -23
- package/dist/report/{react → components/metric-views}/MetricTable.d.ts +3 -3
- package/dist/report/{react → components/metric-views}/MetricTable.js +4 -4
- package/dist/report/{react → components/metric-views}/Scoreboard.d.ts +2 -2
- package/dist/report/{react → components/metric-views}/Scoreboard.js +3 -3
- package/dist/report/components/metric-views/compute.d.ts +89 -0
- package/dist/report/{compute.js → components/metric-views/compute.js} +32 -415
- package/dist/report/components/metric-views/faces.d.ts +13 -0
- package/dist/report/components/metric-views/faces.js +381 -0
- package/dist/report/components/metric-views/index.d.ts +50 -0
- package/dist/report/components/metric-views/index.js +273 -0
- package/dist/report/{text → components/metric-views}/plot.js +1 -1
- package/dist/report/components/shared-compute.d.ts +22 -0
- package/dist/report/components/shared-compute.js +47 -0
- package/dist/report/components/shared-faces.d.ts +9 -0
- package/dist/report/components/shared-faces.js +26 -0
- package/dist/report/components/shared.d.ts +68 -0
- package/dist/report/components/shared.js +125 -0
- package/dist/report/{react → components/site-components}/CopyFixPrompt.d.ts +2 -2
- package/dist/report/{react → components/site-components}/CopyFixPrompt.js +2 -2
- package/dist/report/{react → components/site-components}/HeroCard.d.ts +2 -2
- package/dist/report/{react → components/site-components}/HeroCard.js +2 -2
- package/dist/report/{react → components/site-components}/ScopeWarnings.d.ts +2 -2
- package/dist/report/{react → components/site-components}/ScopeWarnings.js +3 -3
- package/dist/report/{react → components/site-components}/TraceWaterfall.d.ts +3 -3
- package/dist/report/{react → components/site-components}/TraceWaterfall.js +5 -5
- package/dist/report/components/site-components/compute.d.ts +28 -0
- package/dist/report/components/site-components/compute.js +132 -0
- package/dist/report/components/site-components/faces.d.ts +21 -0
- package/dist/report/components/site-components/faces.js +75 -0
- package/dist/report/components/site-components/index.d.ts +74 -0
- package/dist/report/components/site-components/index.js +220 -0
- package/dist/report/{scope-warnings.d.ts → components/site-components/scope-warnings.d.ts} +2 -2
- package/dist/report/{scope-warnings.js → components/site-components/scope-warnings.js} +2 -2
- package/dist/report/{react → components/summaries}/ScopeSummary.d.ts +2 -2
- package/dist/report/{react → components/summaries}/ScopeSummary.js +17 -8
- package/dist/report/components/summaries/compute.d.ts +7 -0
- package/dist/report/components/summaries/compute.js +51 -0
- package/dist/report/components/summaries/faces.d.ts +7 -0
- package/dist/report/components/summaries/faces.js +38 -0
- package/dist/report/components/summaries/index.d.ts +27 -0
- package/dist/report/components/summaries/index.js +78 -0
- package/dist/report/definition/grid-layout.d.ts +50 -0
- package/dist/report/definition/grid-layout.js +89 -0
- package/dist/report/{primitives.d.ts → definition/primitives.d.ts} +34 -4
- package/dist/report/{primitives.js → definition/primitives.js} +101 -10
- package/dist/report/{report.d.ts → definition/report.d.ts} +36 -65
- package/dist/report/{report.js → definition/report.js} +58 -90
- package/dist/report/{text/table.d.ts → definition/table-text.d.ts} +2 -2
- package/dist/report/{text/table.js → definition/table-text.js} +2 -2
- package/dist/report/{tree.d.ts → definition/tree.d.ts} +44 -12
- package/dist/report/{tree.js → definition/tree.js} +10 -12
- package/dist/report/index.d.ts +33 -18
- package/dist/report/index.js +23 -14
- package/dist/report/{aggregate.d.ts → model/aggregate.d.ts} +12 -4
- package/dist/report/{aggregate.js → model/aggregate.js} +28 -8
- package/dist/report/{flag.d.ts → model/flag.d.ts} +14 -1
- package/dist/report/{flag.js → model/flag.js} +35 -0
- package/dist/report/{format.d.ts → model/format.d.ts} +11 -5
- package/dist/report/{format.js → model/format.js} +48 -4
- package/dist/report/{locale.d.ts → model/locale.d.ts} +17 -17
- package/dist/report/{locale.js → model/locale.js} +31 -33
- package/dist/report/{metrics.d.ts → model/metrics.d.ts} +2 -2
- package/dist/report/{metrics.js → model/metrics.js} +2 -2
- package/dist/report/{types.d.ts → model/types.d.ts} +140 -20
- package/dist/report/react/index.d.ts +32 -21
- package/dist/report/react/index.js +33 -21
- package/dist/report/{load.d.ts → runtime/load.d.ts} +1 -1
- package/dist/report/{load.js → runtime/load.js} +1 -1
- package/dist/report/runtime/text.d.ts +67 -0
- package/dist/report/runtime/text.js +114 -0
- package/dist/report/runtime/web.d.ts +35 -0
- package/dist/report/runtime/web.js +70 -0
- package/dist/results/annotated-source.d.ts +87 -0
- package/dist/results/attempt-evidence.d.ts +74 -0
- package/dist/results/attempt-source.d.ts +15 -0
- package/dist/results/select.d.ts +12 -2
- package/dist/results/select.js +27 -5
- package/dist/runner/eval-selection.d.ts +26 -0
- package/dist/runner/feedback/sink.d.ts +26 -1
- package/dist/runner/types.d.ts +149 -23
- package/dist/sandbox/docker.d.ts +6 -0
- package/dist/sandbox/types.d.ts +23 -21
- package/dist/scoring/display.d.ts +7 -2
- package/dist/scoring/display.js +42 -4
- package/dist/scoring/types.d.ts +3 -3
- package/dist/shared/aggregate.d.ts +15 -1
- package/dist/shared/aggregate.js +32 -1
- package/dist/shared/types.d.ts +6 -1
- package/docs-site/images/logo-dark.svg +7 -0
- package/docs-site/images/logo.svg +6 -5
- package/docs-site/zh/README.md +8 -7
- package/docs-site/zh/examples/ai-agent-application.mdx +5 -5
- package/docs-site/zh/examples/coding-agent-extensions.mdx +4 -4
- package/docs-site/zh/examples/index.mdx +3 -3
- package/docs-site/zh/examples/integrations/ai-sdk-v7.mdx +1 -1
- package/docs-site/zh/examples/integrations/claude-sdk.mdx +3 -3
- package/docs-site/zh/examples/integrations/codex-sdk.mdx +3 -3
- package/docs-site/zh/examples/integrations/langgraph.mdx +3 -3
- package/docs-site/zh/examples/integrations/pi-sdk.mdx +2 -2
- package/docs-site/zh/explanation/adapter.mdx +19 -19
- package/docs-site/zh/explanation/assert.mdx +9 -9
- package/docs-site/zh/explanation/drive.mdx +4 -4
- package/docs-site/zh/explanation/evals.mdx +8 -8
- package/docs-site/zh/explanation/experiment.mdx +8 -8
- package/docs-site/zh/explanation/hitl.mdx +13 -13
- package/docs-site/zh/explanation/judge.mdx +3 -3
- package/docs-site/zh/explanation/overview.mdx +7 -7
- package/docs-site/zh/explanation/runner.mdx +11 -11
- package/docs-site/zh/explanation/tier.mdx +6 -6
- package/docs-site/zh/index.mdx +14 -14
- package/docs-site/zh/introduction.mdx +13 -16
- package/docs-site/zh/reference/builtin-agents.mdx +61 -28
- package/docs-site/zh/reference/capabilities.mdx +7 -27
- package/docs-site/zh/reference/cli.mdx +35 -32
- package/docs-site/zh/reference/define-agent.mdx +32 -31
- package/docs-site/zh/reference/define-config.mdx +6 -6
- package/docs-site/zh/reference/define-eval.mdx +33 -20
- package/docs-site/zh/reference/events.mdx +4 -4
- package/docs-site/zh/reference/expect.mdx +3 -3
- package/docs-site/zh/reference/official-adapters.mdx +17 -17
- package/docs-site/zh/reference/report-components.mdx +193 -121
- package/docs-site/zh/reference/results-data.mdx +13 -13
- package/docs-site/zh/troubleshooting/debug-sandbox.mdx +11 -11
- package/docs-site/zh/troubleshooting/debugging.mdx +43 -19
- package/docs-site/zh/{how-to → tutorials}/agent-feedback-loop.mdx +38 -38
- package/docs-site/zh/tutorials/agent-onboarding.mdx +100 -0
- package/docs-site/zh/tutorials/authoring.mdx +211 -0
- package/docs-site/zh/{how-to → tutorials}/ci-integration.mdx +8 -8
- package/docs-site/zh/{how-to → tutorials}/connect-otel.mdx +22 -22
- package/docs-site/zh/{how-to → tutorials}/connect-your-agent.mdx +58 -63
- package/docs-site/zh/tutorials/custom-reports.mdx +453 -0
- package/docs-site/zh/{how-to → tutorials}/dataset-fanout.mdx +4 -4
- package/docs-site/zh/tutorials/experiments.mdx +103 -0
- package/docs-site/zh/{how-to → tutorials}/fixtures.mdx +9 -9
- package/docs-site/zh/{how-to → tutorials}/publish-report.mdx +11 -5
- package/docs-site/zh/tutorials/quickstart.mdx +18 -18
- package/docs-site/zh/{how-to → tutorials}/reporters.mdx +6 -6
- package/docs-site/zh/{how-to → tutorials}/sandbox-agent.mdx +6 -6
- package/docs-site/zh/{how-to → tutorials}/sandbox-providers.mdx +41 -19
- package/docs-site/zh/{how-to → tutorials}/scoring-guide.mdx +4 -4
- package/docs-site/zh/{how-to → tutorials}/viewing-results.mdx +59 -50
- package/docs-site/zh/tutorials/write-experiment.mdx +355 -0
- package/docs-site/zh/{how-to → tutorials}/write-send.mdx +40 -40
- package/package.json +8 -4
- package/src/agents/bub.ts +21 -8
- package/src/agents/claude-code.ts +23 -10
- package/src/agents/codex.test.ts +22 -7
- package/src/agents/codex.ts +22 -9
- package/src/agents/index.ts +2 -2
- package/src/agents/openai-compat.ts +1 -1
- package/src/agents/post-setup.ts +45 -22
- package/src/agents/streaming.ts +1 -1
- package/src/agents/types.ts +20 -17
- package/src/agents/ui-message-stream.test.ts +10 -0
- package/src/agents/ui-message-stream.ts +12 -1
- package/src/cli.ts +77 -64
- package/src/context/types.ts +21 -21
- package/src/define.ts +13 -0
- package/src/i18n/en.ts +30 -15
- package/src/i18n/zh-CN.ts +30 -15
- package/src/index.ts +2 -0
- package/src/report/{react → assets}/colors.ts +22 -0
- package/src/report/{react → assets}/enhance.js +1 -33
- package/src/report/{react → assets}/styles.css +136 -112
- package/src/report/built-in/index.tsx +2 -2
- package/src/report/built-in/standard.tsx +18 -6
- package/src/report/components/attempt-detail/AttemptAssertions.tsx +67 -0
- package/src/report/components/attempt-detail/AttemptConversation.tsx +110 -0
- package/src/report/components/attempt-detail/AttemptDiagnostics.tsx +38 -0
- package/src/report/components/attempt-detail/AttemptDiff.tsx +35 -0
- package/src/report/components/attempt-detail/AttemptError.tsx +31 -0
- package/src/report/components/attempt-detail/AttemptFixPrompt.tsx +26 -0
- package/src/report/components/attempt-detail/AttemptSource.tsx +80 -0
- package/src/report/components/attempt-detail/AttemptSummary.tsx +74 -0
- package/src/report/components/attempt-detail/AttemptTimeline.tsx +108 -0
- package/src/report/components/attempt-detail/AttemptTrace.tsx +63 -0
- package/src/report/components/attempt-detail/AttemptUsage.tsx +29 -0
- package/src/report/components/attempt-detail/attempt-components.test.tsx +680 -0
- package/src/report/components/attempt-detail/compute.ts +287 -0
- package/src/report/components/attempt-detail/faces.ts +257 -0
- package/src/report/components/attempt-detail/index.tsx +583 -0
- package/src/report/components/attempt-detail/validate.test.ts +235 -0
- package/src/report/{react → components}/cell.tsx +6 -3
- package/src/report/{report.test.ts → components/compute.test.ts} +316 -46
- package/src/report/{react → components/entity-lists}/AttemptList.tsx +24 -16
- package/src/report/{react → components/entity-lists}/EvalList.tsx +0 -0
- package/src/report/{react → components/entity-lists}/ExperimentList.tsx +23 -20
- package/src/report/components/entity-lists/compute.ts +196 -0
- package/src/report/components/entity-lists/faces.ts +207 -0
- package/src/report/components/entity-lists/index.tsx +241 -0
- package/src/report/components/entity-lists/validate.test.ts +114 -0
- package/src/report/{react → components}/fixtures.ts +2 -2
- package/src/report/{react → components/metric-views}/DeltaTable.tsx +5 -5
- package/src/report/{react → components/metric-views}/MetricBars.tsx +5 -5
- package/src/report/{react → components/metric-views}/MetricLine.tsx +9 -6
- package/src/report/{react → components/metric-views}/MetricMatrix.tsx +6 -6
- package/src/report/{react → components/metric-views}/MetricScatter.tsx +41 -33
- package/src/report/{react → components/metric-views}/MetricTable.tsx +6 -6
- package/src/report/{react → components/metric-views}/Scoreboard.tsx +4 -4
- package/src/report/{compute.ts → components/metric-views/compute.ts} +51 -477
- package/src/report/components/metric-views/faces.ts +423 -0
- package/src/report/components/metric-views/index.tsx +352 -0
- package/src/report/{text → components/metric-views}/plot.ts +1 -1
- package/src/report/components/metric-views/validate.test.ts +211 -0
- package/src/report/{react → components}/render.test.tsx +47 -17
- package/src/report/components/shared-compute.ts +59 -0
- package/src/report/components/shared-faces.ts +30 -0
- package/src/report/components/shared.ts +192 -0
- package/src/report/{react → components/site-components}/CopyFixPrompt.tsx +3 -3
- package/src/report/{react → components/site-components}/HeroCard.tsx +3 -3
- package/src/report/{react → components/site-components}/ScopeWarnings.tsx +4 -4
- package/src/report/{react → components/site-components}/TraceWaterfall.tsx +13 -10
- package/src/report/components/site-components/compute.ts +144 -0
- package/src/report/components/site-components/faces.ts +80 -0
- package/src/report/components/site-components/index.tsx +265 -0
- package/src/report/{scope-warnings.ts → components/site-components/scope-warnings.ts} +3 -3
- package/src/report/{site-components.test.tsx → components/site-components/site-components.test.tsx} +23 -21
- package/src/report/components/site-components/validate.test.ts +89 -0
- package/src/report/{react → components/summaries}/ScopeSummary.tsx +26 -11
- package/src/report/components/summaries/compute.ts +56 -0
- package/src/report/components/summaries/faces.ts +48 -0
- package/src/report/components/summaries/index.tsx +127 -0
- package/src/report/components/summaries/validate.test.ts +49 -0
- package/src/report/definition/grid-layout.test.ts +124 -0
- package/src/report/definition/grid-layout.ts +146 -0
- package/src/report/{primitives.tsx → definition/primitives.tsx} +168 -18
- package/src/report/{report.ts → definition/report.ts} +98 -165
- package/src/report/{shell-head.test.ts → definition/shell-head.test.ts} +2 -2
- package/src/report/{text/table.ts → definition/table-text.ts} +4 -4
- package/src/report/{tree.ts → definition/tree.ts} +58 -22
- package/src/report/index.ts +126 -66
- package/src/report/{aggregate.ts → model/aggregate.ts} +31 -9
- package/src/report/{flag.ts → model/flag.ts} +39 -1
- package/src/report/{format.ts → model/format.ts} +59 -5
- package/src/report/{locale.ts → model/locale.ts} +34 -36
- package/src/report/{metrics.ts → model/metrics.ts} +3 -3
- package/src/report/{types.ts → model/types.ts} +143 -22
- package/src/report/react/index.tsx +52 -26
- package/src/report/{dual-render.test.tsx → runtime/dual-render.test.tsx} +564 -49
- package/src/report/runtime/host.test.ts +44 -0
- package/src/report/runtime/host.ts +138 -0
- package/src/report/{load.ts → runtime/load.ts} +1 -1
- package/src/report/runtime/text.ts +192 -0
- package/src/report/runtime/web.ts +106 -0
- package/src/results/attempt-evidence.ts +5 -1
- package/src/results/host-equivalence.test.ts +54 -3
- package/src/results/select.ts +30 -6
- package/src/runner/attempt.test.ts +48 -1
- package/src/runner/attempt.ts +62 -42
- package/src/runner/cleanup-timeout.test.ts +16 -0
- package/src/runner/cleanup-timeout.ts +23 -0
- package/src/runner/discover.ts +1 -2
- package/src/runner/eval-selection.test.ts +187 -0
- package/src/runner/eval-selection.ts +109 -0
- package/src/runner/experiment-cleanup-registry.test.ts +89 -0
- package/src/runner/experiment-cleanup-registry.ts +39 -0
- package/src/runner/experiment-labels.test.ts +71 -0
- package/src/runner/feedback/agent.test.ts +33 -1
- package/src/runner/feedback/agent.ts +12 -16
- package/src/runner/feedback/ci.test.ts +35 -1
- package/src/runner/feedback/ci.ts +16 -18
- package/src/runner/feedback/coordinator.ts +28 -0
- package/src/runner/feedback/human.test.ts +108 -1
- package/src/runner/feedback/human.ts +71 -21
- package/src/runner/feedback/reducer.test.ts +64 -0
- package/src/runner/feedback/reducer.ts +26 -0
- package/src/runner/feedback/sink.ts +40 -1
- package/src/runner/fingerprint.ts +2 -1
- package/src/runner/report.test.ts +14 -0
- package/src/runner/report.ts +3 -1
- package/src/runner/run.test.ts +390 -12
- package/src/runner/run.ts +259 -26
- package/src/runner/sandbox-selection.test.ts +13 -3
- package/src/runner/sandbox-selection.ts +4 -2
- package/src/runner/types.ts +158 -23
- package/src/sandbox/docker.ts +7 -0
- package/src/sandbox/e2b.ts +12 -13
- package/src/sandbox/types.ts +23 -21
- package/src/scoring/display.test.ts +28 -2
- package/src/scoring/display.ts +38 -5
- package/src/scoring/types.ts +3 -3
- package/src/shared/aggregate.test.ts +52 -0
- package/src/shared/aggregate.ts +30 -1
- package/src/shared/types.ts +6 -1
- package/src/show/command.test.ts +34 -0
- package/src/show/command.ts +16 -0
- package/src/show/compose.ts +1 -1
- package/src/show/index.ts +58 -22
- package/src/show/render.ts +16 -240
- package/src/show/show.test.ts +128 -35
- package/src/view/app/App.test.tsx +8 -7
- package/src/view/app/App.tsx +135 -51
- package/src/view/app/i18n.ts +7 -147
- package/src/view/app/lib/attempt-dialog.test.ts +65 -0
- package/src/view/app/lib/attempt-dialog.ts +69 -0
- package/src/view/app/main.tsx +0 -1
- package/src/view/app/types.ts +3 -86
- package/src/view/artifact-serving.test.ts +22 -38
- package/src/view/client-dist/app.css +1 -1
- package/src/view/client-dist/app.js +15 -22
- package/src/view/data.test.ts +48 -36
- package/src/view/data.ts +122 -89
- package/src/view/index.ts +3 -13
- package/src/view/server.ts +36 -6
- package/src/view/shared/types.ts +6 -31
- package/src/view/site-parity.test.ts +24 -1
- package/src/view/site.ts +144 -17
- package/src/view/styles.css +18 -806
- package/src/view/view-report.test.ts +182 -34
- package/dist/report/components.d.ts +0 -179
- package/dist/report/components.js +0 -544
- package/dist/report/compute.d.ts +0 -139
- package/dist/report/react/ExperimentComparison.d.ts +0 -10
- package/dist/report/react/ExperimentComparison.js +0 -12
- package/dist/report/react/MetricScatter.d.ts +0 -9
- package/dist/report/react/format.d.ts +0 -3
- package/dist/report/react/format.js +0 -7
- package/dist/report/text/faces.d.ts +0 -45
- package/dist/report/text/faces.js +0 -671
- package/dist/report/web.d.ts +0 -32
- package/dist/report/web.js +0 -48
- package/docs-site/zh/how-to/authoring.mdx +0 -162
- package/docs-site/zh/how-to/custom-reports.mdx +0 -414
- package/docs-site/zh/how-to/experiments.mdx +0 -86
- package/docs-site/zh/how-to/write-experiment.mdx +0 -164
- package/src/report/components.tsx +0 -925
- package/src/report/react/ExperimentComparison.tsx +0 -73
- package/src/report/react/format.ts +0 -9
- package/src/report/text/faces.ts +0 -767
- package/src/report/web.ts +0 -77
- package/src/show/report-host.test.ts +0 -205
- package/src/show/report-host.ts +0 -389
- package/src/view/app/components/AttemptModal.tsx +0 -496
- package/src/view/app/components/CodeView.test.tsx +0 -142
- package/src/view/app/components/CodeView.tsx +0 -310
- package/src/view/app/components/CopyControls.tsx +0 -106
- package/src/view/app/components/Trace.tsx +0 -100
- package/src/view/app/components/Transcript.tsx +0 -157
- package/src/view/app/components/ui/badge.tsx +0 -21
- package/src/view/app/lib/artifact-url.ts +0 -17
- package/src/view/app/lib/attempt-route.test.ts +0 -80
- package/src/view/app/lib/attempt-route.ts +0 -52
- package/src/view/app/lib/format.ts +0 -70
- package/src/view/app/lib/guards.test.ts +0 -108
- package/src/view/app/lib/guards.ts +0 -71
- package/src/view/app/lib/rows.ts +0 -22
- package/src/view/app/lib/transcript-data.tsx +0 -151
- package/src/view/app/lib/verdict.ts +0 -23
- package/src/view/app/shared.ts +0 -8
- /package/dist/report/{react → components/metric-views}/chart-math.d.ts +0 -0
- /package/dist/report/{react → components/metric-views}/chart-math.js +0 -0
- /package/dist/report/{text → components/metric-views}/plot.d.ts +0 -0
- /package/dist/report/{react → components/site-components}/PoweredBy.d.ts +0 -0
- /package/dist/report/{react → components/site-components}/PoweredBy.js +0 -0
- /package/dist/report/{text/layout.d.ts → model/text-layout.d.ts} +0 -0
- /package/dist/report/{text/layout.js → model/text-layout.js} +0 -0
- /package/dist/report/{types.js → model/types.js} +0 -0
- /package/src/report/{react → components/metric-views}/chart-math.test.ts +0 -0
- /package/src/report/{react → components/metric-views}/chart-math.ts +0 -0
- /package/src/report/{react → components/site-components}/PoweredBy.tsx +0 -0
- /package/src/report/{text/layout.ts → model/text-layout.ts} +0 -0
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: "用 niceeval/results 直接读写结果数据"
|
|
3
3
|
sidebarTitle: "结果数据 API"
|
|
4
|
-
description: "报告积木脚下的数据层:openResults 把 .niceeval/ 的落盘 artifact parse 成「实验 → 结果快照 →
|
|
4
|
+
description: "报告积木脚下的数据层:openResults 把 .niceeval/ 的落盘 artifact parse 成「实验 → 结果快照 → 评估 → attempt」的类型化层次,createResultsWriter 把别家结果写成 NiceEval 格式,copySnapshots 负责发布瘦身。"
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
[自定义报告](/zh/
|
|
7
|
+
[自定义报告](/zh/tutorials/custom-reports)的积木——指标、计算函数、双面组件——脚下还有一层:`niceeval/results`,落盘 artifact 的 parser。官方两扇门和全部报告积木读的都是这一层,没有私有数据通道;报告表达不了的口径,下到这层直接拿数据算。
|
|
8
8
|
|
|
9
9
|
什么时候下到这层:
|
|
10
10
|
|
|
@@ -16,7 +16,7 @@ description: "报告积木脚下的数据层:openResults 把 .niceeval/ 的落
|
|
|
16
16
|
|
|
17
17
|
## 读:`openResults`
|
|
18
18
|
|
|
19
|
-
输入是 `.niceeval/` 目录(或 `copySnapshots` 产出的结果根目录,同一种布局),输出是四层数据:**实验 → 结果快照(单次跑的实验)→
|
|
19
|
+
输入是 `.niceeval/` 目录(或 `copySnapshots` 产出的结果根目录,同一种布局),输出是四层数据:**实验 → 结果快照(单次跑的实验)→ 评估 → attempt**。你从此不碰路径、不判断文件存在性、不解析 JSON:
|
|
20
20
|
|
|
21
21
|
```typescript
|
|
22
22
|
import { openResults } from "niceeval/results";
|
|
@@ -51,7 +51,7 @@ snap.producer; // { name: "niceeval", version: "0.4.6" } ——
|
|
|
51
51
|
snap.schemaVersion; // 结果格式版本
|
|
52
52
|
```
|
|
53
53
|
|
|
54
|
-
|
|
54
|
+
**第三层,评估**——这次实验里的每道题,attempt(重试历史)挂在题下面:
|
|
55
55
|
|
|
56
56
|
```typescript
|
|
57
57
|
for (const ev of snap.evals) {
|
|
@@ -134,7 +134,7 @@ for (const s of results.skipped) {
|
|
|
134
134
|
|
|
135
135
|
## 选快照:`results.latest()`
|
|
136
136
|
|
|
137
|
-
|
|
137
|
+
多数场景先回答「每个实验现在的最新结果是什么」,选择器替你挑「每个实验最新一次」——这也是默认报告的口径。返回的是一个 **Scope**,快照和警告绑在一起走:
|
|
138
138
|
|
|
139
139
|
```typescript
|
|
140
140
|
const latest = results.latest({
|
|
@@ -158,9 +158,9 @@ latest.warnings[0];
|
|
|
158
158
|
// }
|
|
159
159
|
```
|
|
160
160
|
|
|
161
|
-
字段供程序判断——CI 里「覆盖缩水就 fail」直接判 `covered < total`,不解析文本;`message` 是渲染好的英文句子,要展示就原样打。渲染与否在你,缺口永远被算出来。警告不止这一种:快照落后于
|
|
161
|
+
字段供程序判断——CI 里「覆盖缩水就 fail」直接判 `covered < total`,不解析文本;`message` 是渲染好的英文句子,要展示就原样打。渲染与否在你,缺口永远被算出来。警告不止这一种:快照落后于 Scope 中最新的落盘进 `stale-snapshot`、选中的快照没收尾(进程中断)进 `unfinished-snapshot`,每种都带 `kind`、可判断的结构化字段和渲染好的 `message`。
|
|
162
162
|
|
|
163
|
-
|
|
163
|
+
Scope 是[报告积木](/zh/tutorials/custom-reports)和下文 `copySnapshots` 的通用输入:收 `Scope` 时 warnings 随行(`ScopeWarnings` 组件会如实展示),手工挑的 `Snapshot[]` 数组照收。微调官方口径不用降级成裸数组:`latest.filter((s) => s.experimentId !== "compare/broken")` 返回新 Scope——快照被删减,warnings 修剪到幸存的实验,provenance 不丢。`filter` 只做删减;「换成该实验上一个完整快照」这类**替换式**重挑不是它的事,回到 `exp.snapshots` 自己拿——手工挑的数组没有挑选过程,自然没有 warnings 可带,也如实。
|
|
164
164
|
|
|
165
165
|
## 一个真实脚本:分布不是折叠
|
|
166
166
|
|
|
@@ -184,9 +184,9 @@ for (const exp of results.experiments) {
|
|
|
184
184
|
}
|
|
185
185
|
```
|
|
186
186
|
|
|
187
|
-
即使在这条最深的路径上也不碰磁盘布局:路径拼接、存在性、版本过滤、快照切分全被库消化。要把这份分布摆进报告页,用 [`defineComponent`](/zh/
|
|
187
|
+
即使在这条最深的路径上也不碰磁盘布局:路径拼接、存在性、版本过滤、快照切分全被库消化。要把这份分布摆进报告页,用 [`defineComponent`](/zh/tutorials/custom-reports) 包一个双面组件即可。
|
|
188
188
|
|
|
189
|
-
一条跨快照累计时的义务:NiceEval 默认把上一轮已有确定判定(passed / failed
|
|
189
|
+
一条跨快照累计时的义务:NiceEval 默认把上一轮已有确定判定(passed / failed)、且评估用例代码和配置没变的结果携带合入新快照(`--force` 全部重跑),同一个 attempt 因此可能存在于多份落盘。携带条目不是空壳:它带着原快照的 `startedAt`(身份锚)与 `artifactBase`(指向原快照 attempt 目录的相对路径),懒加载按候选顺序回退——先本快照的 attempt 目录,再 `artifactBase` 指向的原快照目录(原快照被清理后如实返回 `null`);`ref` 指向条目所在的落盘,即携带入的那份新快照。身份键 `(experimentId, evalId, attempt, startedAt)` 的四个字段都在数据上——前两个是 attempt 的直达字段,序号与 `startedAt` 在 `attempt.result` 上。reader 忠实反映这份重复;跨快照聚合前用 `dedupeAttempts` 按身份键去重,重复保留最新快照里的那份——报告积木的计算函数内置这条,自己写脚本时记得过一遍:
|
|
190
190
|
|
|
191
191
|
```typescript
|
|
192
192
|
import { dedupeAttempts } from "niceeval/results";
|
|
@@ -223,7 +223,7 @@ for (const r of convertedResults) {
|
|
|
223
223
|
await writer.finish(); // 给每个快照补 completedAt,没有任何收尾聚合
|
|
224
224
|
```
|
|
225
225
|
|
|
226
|
-
`writer.snapshot()` 就是读取面「实验 → 快照」层次的镜像:转多个 experiment 就开多个快照目录,experimentId / agent / model / startedAt 这些快照级元数据在这里声明一次,不用塞进每条 attempt;可选的 `knownEvalIds
|
|
226
|
+
`writer.snapshot()` 就是读取面「实验 → 快照」层次的镜像:转多个 experiment 就开多个快照目录,experimentId / agent / model / startedAt 这些快照级元数据在这里声明一次,不用塞进每条 attempt;可选的 `knownEvalIds`(该实验已知的评估用例并集)也在这里声明——它是残缺检测的分母,转换只覆盖部分题目时如实交代全集,下游的覆盖警告就能算出来(`copySnapshots` 发布时会自动补记这个字段,见下文)。转完的目录就是标准结果目录:`niceeval show` / `niceeval view` 直接能看,报告积木直接能算,不用抄格式文档;`producer` 会原样出现在读取面的 `snap.producer` 上。**每个文件恰好写入一次**是写入面的核心承诺:`snapshot.json` 开跑即写、收尾只补 `completedAt`;`result.json` 与 artifact 随 attempt 完成落盘。进程中断只丢未完成的 attempt,已完成的判定与 artifact 已经在盘上——真正「有 attempt 落盘却没有 `snapshot.json`」的极端情况才归 `skipped("incomplete")`,未收尾但元数据齐全的快照能正常读,只带一条警告。
|
|
227
227
|
|
|
228
228
|
## 发布:`copySnapshots`
|
|
229
229
|
|
|
@@ -240,18 +240,18 @@ await copySnapshots(results.latest(), "site-data/run", {
|
|
|
240
240
|
// 读 o11y 的指标就把它带上,不然渲染成「—」
|
|
241
241
|
```
|
|
242
242
|
|
|
243
|
-
第一个参数收 `
|
|
243
|
+
第一个参数收 `Scope` 或手工挑的 `Snapshot[]`——和报告积木同一个输入约定。`artifacts` 的合法值是 `"events" | "trace" | "o11y" | "agentSetup" | "diff" | "sources"`;缺省带除 `diff` 外的五类。目标目录已存在且非空时报错,不静默覆盖——发布脚本要幂等就自己先清目标目录。
|
|
244
244
|
|
|
245
245
|
复制开始前,NiceEval 会规划全部目标文件并检查序列化后的大小。任一文件超过固定的 50 MiB,整次复制在创建目标目录前失败,错误会列出路径、实际大小和处理建议。你可以从 `artifacts` 排除那类证据;如果是旧版本留下的超大 events / trace,用当前版本重跑后再发布。这个检查既覆盖没有逐值截断的源码 / diff,也覆盖单值都正常但累计过大的 JSON,避免直到 `git push` 才撞上 Git host 的单文件限制。
|
|
246
246
|
|
|
247
|
-
大小预检只决定整次复制成功或失败,不会从一个超大文件中间删内容。复制忠实于源:artifact 按原字节复制,不重新序列化、不改写。唯一随行补记的是挑选时的**覆盖事实**:`partial-coverage` 警告的分母是实验的历史并集,而发布目录没有历史——所以每个复制出的快照带上 `knownEvalIds
|
|
247
|
+
大小预检只决定整次复制成功或失败,不会从一个超大文件中间删内容。复制忠实于源:artifact 按原字节复制,不重新序列化、不改写。唯一随行补记的是挑选时的**覆盖事实**:`partial-coverage` 警告的分母是实验的历史并集,而发布目录没有历史——所以每个复制出的快照带上 `knownEvalIds`(复制时刻该实验已知的评估用例并集),reader 端把它并进 `exp.evalIds` 的计算(取本地历史与快照携带值的并集)。发布目录上重新 `openResults().latest()`,残缺警告被同一套机制重新算出来,不靠发布者转述。复制出的目录就是标准结果目录,`niceeval view --results <目录>` 直接能看;要让报告站随 push 自动更新,workflow 见[通过 CI 发布报告](/zh/tutorials/publish-report)。
|
|
248
248
|
|
|
249
249
|
## 分层速览
|
|
250
250
|
|
|
251
251
|
| 层 | 入口 | 回答 |
|
|
252
252
|
| --- | --- | --- |
|
|
253
253
|
| 官方两扇门 | `niceeval show` / `niceeval view` | 零代码看官方摆法 |
|
|
254
|
-
| 报告积木 | [自定义报告](/zh/
|
|
254
|
+
| 报告积木 | [自定义报告](/zh/tutorials/custom-reports)、[报告组件](/zh/reference/report-components) | 自己的口径与摆法 |
|
|
255
255
|
| 结果数据 API(本页) | `niceeval/results` | 折叠表达不了的算法、接自己的系统、读写格式本身 |
|
|
256
256
|
|
|
257
257
|
每层都建立在下一层之上,同一份落盘 artifact 是唯一事实来源——上层的派生物删了随时可重算。
|
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "
|
|
3
|
-
sidebarTitle: "
|
|
4
|
-
description: "用 --keep-sandbox 把失败 Attempt
|
|
2
|
+
title: "保留 Sandbox 现场排查问题"
|
|
3
|
+
sidebarTitle: "保留 Sandbox 现场"
|
|
4
|
+
description: "用 --keep-sandbox 把失败 Attempt 的 Sandbox 保留成可随时唤醒的现场,用 niceeval sandbox enter 进去手动排查,用 sandbox list / stop 查看和清理。"
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Sandbox 默认在每个 Attempt 结束后销毁,排查依据是落盘的 artifact:`niceeval show` 能看到判定、断言、diff 和事件流。大多数问题到这里就够了——完整的排查路线见 [Debug 手册](/zh/troubleshooting/debugging)。
|
|
8
8
|
|
|
9
9
|
但有些问题只能进活的环境里看:
|
|
10
10
|
|
|
11
|
-
- **环境起不来**——setup 阶段装依赖失败、agent CLI 启动不了。这时 agent
|
|
11
|
+
- **环境起不来**——setup 阶段装依赖失败、agent CLI 启动不了。这时 agent 还没开始跑,事件流是空的,最快的办法是进 Sandbox 手动重跑一遍安装命令。
|
|
12
12
|
- **改动落在 `git diff` 之外**——全局装了什么包、`$HOME` 下写了什么配置、`PATH` 实际是什么,artifact 里没有。
|
|
13
13
|
- **重跑太慢**——冷启动加安装要几分钟,想逐条验证猜测时,留着现场比每次重跑快得多。
|
|
14
14
|
|
|
@@ -19,9 +19,9 @@ npx niceeval exp local onboarding/tool-first --keep-sandbox # 等价 --ke
|
|
|
19
19
|
npx niceeval exp local onboarding/tool-first --keep-sandbox=all # 通过的也保留
|
|
20
20
|
```
|
|
21
21
|
|
|
22
|
-
`--keep-sandbox` 是 `niceeval exp` 的运行参数,两档:`failed`(缺省值)保留判定为 `failed` 或 `errored` 的 Attempt(包括超时打断的);`all` 连通过的也保留——调 setup
|
|
22
|
+
`--keep-sandbox` 是 `niceeval exp` 的运行参数,两档:`failed`(缺省值)保留判定为 `failed` 或 `errored` 的 Attempt(包括超时打断的);`all` 连通过的也保留——调 setup Hook、核对通过环境的真实状态时用它,不用故意弄挂一条评估用例。不带这个参数时全部销毁。
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
运行结束后,摘要里会列出保留了哪些 Sandbox、怎么进去:
|
|
25
25
|
|
|
26
26
|
```text
|
|
27
27
|
Kept sandboxes (1)
|
|
@@ -30,11 +30,11 @@ Kept sandboxes (1)
|
|
|
30
30
|
Stop them with: niceeval sandbox stop --all
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
每行给三样东西:Attempt 定位符(用 `niceeval show @1x7f3q9k`
|
|
33
|
+
每行给三样东西:Attempt 定位符(用 `niceeval show @1x7f3q9k` 看落盘证据)、Sandbox 实例 id、进入现场的命令。保留下来的 Sandbox 不会一直跑着烧资源——Docker 容器停在磁盘上,E2B 微 VM 暂停计费,Vercel 保存文件系统。`niceeval sandbox enter` 会先唤醒再进入,在 workdir 打开 shell;退出 shell 后现场自动回到休眠(想让它保持运行,加 `--leave-running`)。进去之后就是这次 Attempt 跑完时的环境,可以手动执行命令、翻文件、复现失败。
|
|
34
34
|
|
|
35
35
|
## 查看和清理
|
|
36
36
|
|
|
37
|
-
|
|
37
|
+
保留下来的 Sandbox 逐条记录在 `.niceeval/sandboxes/` 里,用 `niceeval sandbox` 管理:
|
|
38
38
|
|
|
39
39
|
```bash
|
|
40
40
|
niceeval sandbox list # 列出保留的沙箱和现场状态
|
|
@@ -43,7 +43,7 @@ niceeval sandbox stop a3f9c2d1 # 销毁指定沙箱(id 可以只写唯一
|
|
|
43
43
|
niceeval sandbox stop --all # 全部销毁
|
|
44
44
|
```
|
|
45
45
|
|
|
46
|
-
`stop`
|
|
46
|
+
`stop` 是幂等的:Sandbox 已经不在了(手动删过、云端过期)不算错误,只会把记录移掉并说明。如果 provider 销毁失败,命令会保留记录并返回错误,方便稍后重试,不会把仍活着的资源从列表里藏掉。忘了清也有提醒——下次运行开始时,如果还有上次保留的 Sandbox,会打一行提示。
|
|
47
47
|
|
|
48
48
|
## 各 Provider 的差别
|
|
49
49
|
|
|
@@ -54,4 +54,4 @@ niceeval sandbox stop --all # 全部销毁
|
|
|
54
54
|
|
|
55
55
|
## 边界
|
|
56
56
|
|
|
57
|
-
|
|
57
|
+
保留的 Sandbox 只用来排查,不能续跑或重新评分;判定、断言、diff 这些结论仍以 artifact 为准。查看 artifact 的方法见[查看结果](/zh/tutorials/viewing-results)。
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: "排查失败与复盘历史运行"
|
|
3
3
|
sidebarTitle: "Debug 手册"
|
|
4
|
-
description: "
|
|
4
|
+
description: "一份按场景组织的排查手册:断言失败怎么定位、环境错误怎么进 Sandbox、agent 改了什么怎么看、旧的运行怎么翻出来复盘——每一步都有命令顺序和输出示例。"
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
跑完一次 `niceeval exp`,失败的 Attempt 都带一个 `@` 开头的定位符(如 `@1qrdcfq8`)。它出现在运行摘要、CI 日志和报告里,定位符本身不会过期——只要 `.niceeval/` 里对应的结果快照还在,今天的定位符下周还能用同一条命令打开同一次 Attempt。所有排查都从它开始。
|
|
@@ -34,7 +34,7 @@ timing: sandbox.queue 0.2s · sandbox.create 5.6s · sandbox.setup 3.5s · agent
|
|
|
34
34
|
changes: 2 files changed by agent · M manager_decisions.json · A notes/decision-log.md
|
|
35
35
|
|
|
36
36
|
available:
|
|
37
|
-
niceeval show @1qrdcfq8 --
|
|
37
|
+
niceeval show @1qrdcfq8 --source
|
|
38
38
|
niceeval show @1qrdcfq8 --execution
|
|
39
39
|
niceeval show @1qrdcfq8 --timing
|
|
40
40
|
niceeval show @1qrdcfq8 --diff
|
|
@@ -46,10 +46,10 @@ available:
|
|
|
46
46
|
|
|
47
47
|
排查顺序是「哪条断言挂了 → agent 当时做了什么 → 它到底改了什么」。
|
|
48
48
|
|
|
49
|
-
**1. 把断言放回源码。** `--
|
|
49
|
+
**1. 把断言放回源码。** `--source` 显示运行时保存的那份评估用例源码(不是你工作区里可能已经改过的版本),失败的断言直接标在对应行上;`t.send(...)` 的调用行标出它产生的那一轮——轮标签(`s1/t1`,与 `--execution` / `--timing` 用同一套)、这轮成没成、花了多久:
|
|
50
50
|
|
|
51
51
|
```text
|
|
52
|
-
$ niceeval show @1qrdcfq8 --
|
|
52
|
+
$ niceeval show @1qrdcfq8 --source
|
|
53
53
|
21✓ await t.send("Review the proposals and record your decision…");
|
|
54
54
|
s1/t1 · completed · 22.4s
|
|
55
55
|
38 for (const [issue, label] of Object.entries(expected)) {
|
|
@@ -80,6 +80,20 @@ TURN s1/t1 · completed · 22.4s · 12.4k tok · $0.02
|
|
|
80
80
|
|
|
81
81
|
对话按轮分段,每轮头行给出编号(`s1/t1`)、状态、耗时和用量——这个编号和 `--diff`、`--timing` 里的轮次标签是同一套,能互相对照。
|
|
82
82
|
|
|
83
|
+
不想通读全文时,接 `grep` 定向查。列出这次 Attempt 用过哪些工具、各多少次:
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
$ niceeval show @1qrdcfq8 --execution | grep "TOOL ·" | sort | uniq -c
|
|
87
|
+
5 TOOL · command_execution
|
|
88
|
+
2 TOOL · file_change
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
想确认它有没有跑过某条命令、有没有提到某个文件,直接搜关键词:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
niceeval show @1qrdcfq8 --execution | grep proposals
|
|
95
|
+
```
|
|
96
|
+
|
|
83
97
|
**3. 看它到底改了什么。** `--diff` 只显示 **agent 自己改动的文件**——你上传的起始文件、跑完后写入的验证材料不会混在里面,所以列表里的每一行都真的是 agent 干的:
|
|
84
98
|
|
|
85
99
|
```text
|
|
@@ -104,7 +118,7 @@ M manager_decisions.json · changed in s1/t1, s1/t2
|
|
|
104
118
|
|
|
105
119
|
到这里通常能下结论:是任务描述有歧义、agent 理解错了,还是断言本身写得太死。
|
|
106
120
|
|
|
107
|
-
**要看文件本身,而不只是改动?** 落盘的证据刻意不保存整个工作区——`--diff` 只有 agent 改过的文件,agent 该写没写的文件、你上传的起始材料、setup
|
|
121
|
+
**要看文件本身,而不只是改动?** 落盘的证据刻意不保存整个工作区——`--diff` 只有 agent 改过的文件,agent 该写没写的文件、你上传的起始材料、setup 装出来的东西都不在里面。想看它们的实际内容,进活现场:重跑这一条评估用例加 `--keep-sandbox`(`failed` 的 Attempt 同样会保留,不只是环境错误),用下面场景二的方式进 Sandbox,workdir 里就是这次跑完时的完整文件树。
|
|
108
122
|
|
|
109
123
|
## 场景二:环境错误(errored)——agent 根本没跑起来
|
|
110
124
|
|
|
@@ -126,9 +140,9 @@ timing: sandbox.queue 1.2s · sandbox.create 2m 6s ✗ failed here
|
|
|
126
140
|
|
|
127
141
|
`phase` 直接告诉你死在哪一步,而且决定了下一步走哪条路:
|
|
128
142
|
|
|
129
|
-
**`sandbox.create`
|
|
143
|
+
**`sandbox.create` 失败——Sandbox 根本没创建出来,没有现场可留。** 这类错误(配额、限流、凭据、镜像 / 模板不存在)在你自己的机器和账号侧排查:核对 API key 和配额、降低 `--max-concurrency`、确认镜像 / 模板名。示例里的 rate-limit 就属于这类,重跑加 `--keep-sandbox` 只会原地再死一次。
|
|
130
144
|
|
|
131
|
-
**`sandbox.setup` / `agent.setup` / `eval.run`
|
|
145
|
+
**`sandbox.setup` / `agent.setup` / `eval.run` 失败——Sandbox 活过,值得留现场。** 装依赖失败、agent CLI 起不来、跑到一半超时,这类问题事件流往往是空的,落盘证据帮不上忙,最快的办法是留住现场进去手动重跑一遍出错的命令:
|
|
132
146
|
|
|
133
147
|
```bash
|
|
134
148
|
# 只重跑这一条 eval,失败时保留沙箱
|
|
@@ -142,11 +156,11 @@ Kept sandboxes (1)
|
|
|
142
156
|
Stop them with: niceeval sandbox stop --all
|
|
143
157
|
```
|
|
144
158
|
|
|
145
|
-
`niceeval sandbox enter a3f9c2d1` 会唤醒现场并在 workdir 打开 shell——手动执行安装命令看真实报错、翻 `$HOME` 下的配置、检查 `PATH`,这些都在 artifact 之外,只有活现场能回答;退出 shell 后现场自动回到休眠,不白烧资源。保留策略、各 provider 的差别见[
|
|
159
|
+
`niceeval sandbox enter a3f9c2d1` 会唤醒现场并在 workdir 打开 shell——手动执行安装命令看真实报错、翻 `$HOME` 下的配置、检查 `PATH`,这些都在 artifact 之外,只有活现场能回答;退出 shell 后现场自动回到休眠,不白烧资源。保留策略、各 provider 的差别见[保留 Sandbox 现场](/zh/troubleshooting/debug-sandbox)。
|
|
146
160
|
|
|
147
|
-
##
|
|
161
|
+
## 查看和清理留下的 Sandbox
|
|
148
162
|
|
|
149
|
-
|
|
163
|
+
保留下来的 Sandbox 不会一直烧资源:Docker 容器停驻在磁盘上,E2B 微 VM 暂停计费,进入时自动唤醒。用 `niceeval sandbox` 管理:
|
|
150
164
|
|
|
151
165
|
```text
|
|
152
166
|
$ niceeval sandbox list
|
|
@@ -168,17 +182,26 @@ niceeval sandbox stop --all
|
|
|
168
182
|
|
|
169
183
|
每次运行都会在 `.niceeval/<实验>/<时间戳>/` 下留一份完整的结果快照,判定、断言、事件流、diff 都在里面,**不会被下一次运行覆盖**。复盘有三个入口:
|
|
170
184
|
|
|
171
|
-
**用旧定位符直接打开。** 从上周的终端记录、CI 日志或报告里复制 `@` 定位符,`niceeval show @<定位符>` 照常工作,上面的 `--
|
|
185
|
+
**用旧定位符直接打开。** 从上周的终端记录、CI 日志或报告里复制 `@` 定位符,`niceeval show @<定位符>` 照常工作,上面的 `--source` / `--execution` / `--diff` 全部可用——包括那份运行时的评估用例源码,哪怕你后来把评估用例改了。
|
|
172
186
|
|
|
173
|
-
|
|
187
|
+
**按实验回看现在的判定。** 不记得定位符时,从实验入手:
|
|
174
188
|
|
|
175
189
|
```bash
|
|
176
|
-
niceeval show --
|
|
177
|
-
niceeval show memory/swelancer --
|
|
190
|
+
niceeval show --exp compare/bub # 这个实验每道题现在的判定
|
|
191
|
+
niceeval show memory/swelancer --exp compare/bub # 收窄到某道题
|
|
178
192
|
```
|
|
179
193
|
|
|
180
194
|
列表里每道题、每次 Attempt 都带定位符,接着往深处钻就回到上面的场景一 / 场景二。
|
|
181
195
|
|
|
196
|
+
**同一个检查刷一批 Attempt。** 从实验列表里复制定位符,套一层循环。比如核对这几次 Attempt 里谁调用过 `file_change`:
|
|
197
|
+
|
|
198
|
+
```bash
|
|
199
|
+
for loc in @1qrdcfq8 @1x7f3q8a @18c1m2qx; do
|
|
200
|
+
echo "== $loc"
|
|
201
|
+
niceeval show $loc --execution | grep "TOOL · file_change"
|
|
202
|
+
done
|
|
203
|
+
```
|
|
204
|
+
|
|
182
205
|
**在浏览器里翻。** 复盘一批失败、对比多次 Attempt 时,网页比终端顺手:
|
|
183
206
|
|
|
184
207
|
```bash
|
|
@@ -200,13 +223,14 @@ niceeval view --results site-data/run
|
|
|
200
223
|
|
|
201
224
|
| 症状 | 命令顺序 |
|
|
202
225
|
|---|---|
|
|
203
|
-
| 断言挂了,不知道为什么 | `show @loc` → `show @loc --
|
|
226
|
+
| 断言挂了,不知道为什么 | `show @loc` → `show @loc --source` |
|
|
204
227
|
| 想知道 agent 当时做了什么 | `show @loc --execution` |
|
|
228
|
+
| 想确认有没有调用过某个工具 / 搜对话关键词 | `show @loc --execution \| grep "TOOL ·" \| sort \| uniq -c` / `\| grep <关键词>` |
|
|
205
229
|
| 想确认 agent 改了哪些文件 | `show @loc --diff` → `--diff=<path>` |
|
|
206
230
|
| 哪一步慢 / 超时死在哪 | `show @loc`(看 `timing:` 行)→ `show @loc --timing` |
|
|
207
|
-
|
|
|
208
|
-
| 装依赖失败、CLI 起不来、跑一半超时 |
|
|
231
|
+
| Sandbox 创建就失败(配额 / 凭据 / 镜像) | `show @loc` 看 error 的 code 与 cause → 查账号配额、核对凭据、降 `--max-concurrency`(没有现场可留) |
|
|
232
|
+
| 装依赖失败、CLI 起不来、跑一半超时 | 重跑该评估用例加 `--keep-sandbox` → `niceeval sandbox enter <id>` |
|
|
209
233
|
| 想看文件实际内容(agent 没改的、起始材料、`$HOME`) | 重跑加 `--keep-sandbox`(failed 也留)→ `sandbox enter` 进 workdir 看 |
|
|
210
|
-
|
|
|
211
|
-
| 复盘上周那次失败 | 翻出旧定位符 → `show @loc`;不记得就 `show --
|
|
234
|
+
| 留了哪些 Sandbox、清理 | `sandbox list` → `sandbox stop <id>` / `--all` |
|
|
235
|
+
| 复盘上周那次失败 | 翻出旧定位符 → `show @loc`;不记得就 `show --exp <实验>` |
|
|
212
236
|
| 一批失败一起看 | `niceeval view` → Attempt 详情 → Copy fix prompt |
|
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "
|
|
3
|
-
sidebarTitle: "
|
|
4
|
-
description: "让
|
|
2
|
+
title: "用 Coding Agent 编写评估用例并迭代程序"
|
|
3
|
+
sidebarTitle: "Coding Agent 反馈闭环"
|
|
4
|
+
description: "让 Coding Agent 读取随包文档,用 Bash 运行实验、查看结果、定位失败并持续迭代,最后通过全量重跑确认结果。"
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
Coding
|
|
7
|
+
Coding Agent 可以根据需求编写 NiceEval 配置与评估用例,也可以根据运行结果持续改进被测程序。整个过程只需要仓库文件和 Bash,不依赖浏览器或额外的 Skill。
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## 读取安装版本的文档
|
|
10
10
|
|
|
11
|
-
NiceEval 把中文文档发布在 npm 包的 `docs-site/zh/`
|
|
11
|
+
NiceEval 把中文文档发布在 npm 包的 `docs-site/zh/` 目录,并在包根提供供 Coding Agent 使用的 `INDEX.md`。Coding Agent 应先读取 `node_modules/niceeval/INDEX.md`,再从索引进入当前任务需要的页面,不依赖训练数据或其它版本的在线示例。这样可以保证 API、CLI 与安装版本一致。
|
|
12
12
|
|
|
13
13
|
`npx niceeval init` 会初始化配置,并把一段托管指引写进项目的 `AGENTS.md`。如果项目只有 `CLAUDE.md`,则写入 `CLAUDE.md`;两份文件都不存在时,新建 `AGENTS.md`。升级 NiceEval 后再运行一次 `init`,即可刷新托管区块。
|
|
14
14
|
|
|
@@ -25,15 +25,15 @@ AI 通常按任务选择这些入口:
|
|
|
25
25
|
| 任务 | 随包文档 |
|
|
26
26
|
| --- | --- |
|
|
27
27
|
| 初始化项目 | `docs-site/zh/tutorials/quickstart.mdx` |
|
|
28
|
-
|
|
|
29
|
-
| 定义实验 | `docs-site/zh/
|
|
28
|
+
| 编写评估用例 | `docs-site/zh/explanation/evals.mdx` |
|
|
29
|
+
| 定义实验 | `docs-site/zh/tutorials/write-experiment.mdx` |
|
|
30
30
|
| 连接被测 Agent | `docs-site/zh/explanation/adapter.mdx` |
|
|
31
|
-
| 配置 Sandbox | `docs-site/zh/
|
|
32
|
-
| 解释运行结果 | `docs-site/zh/
|
|
31
|
+
| 配置 Sandbox | `docs-site/zh/tutorials/sandbox-agent.mdx` |
|
|
32
|
+
| 解释运行结果 | `docs-site/zh/tutorials/viewing-results.mdx` |
|
|
33
33
|
|
|
34
34
|
## 用 bash 完成一次反馈闭环
|
|
35
35
|
|
|
36
|
-
|
|
36
|
+
闭环的目标不是”把命令跑绿”,而是形成并验证一个假设:失败来自被测程序、评估用例,还是运行环境。推荐按下面的顺序迭代。
|
|
37
37
|
|
|
38
38
|
<Steps>
|
|
39
39
|
<Step title="运行实验">
|
|
@@ -41,7 +41,7 @@ AI 通常按任务选择这些入口:
|
|
|
41
41
|
npx niceeval exp local --output agent
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
先读取退出码:`0`
|
|
44
|
+
先读取退出码:`0` 表示所有评估用例通过;`1` 表示至少一个评估用例失败或出错;`2` 表示 NiceEval 自身未能完成运行。退出码决定是否继续,控制台文本用于定位原因。
|
|
45
45
|
</Step>
|
|
46
46
|
<Step title="读取失败">
|
|
47
47
|
```bash
|
|
@@ -49,7 +49,7 @@ AI 通常按任务选择这些入口:
|
|
|
49
49
|
npx niceeval show @1k2m9qtr
|
|
50
50
|
```
|
|
51
51
|
|
|
52
|
-
|
|
52
|
+
第一条命令显示当前各实验的通过率、成本、耗时,以及每个评估用例的紧凑 Attempt locator——locator 本身就是证据入口,不在列表里编码证据可用性。第二条命令直接打开选中的 Attempt,页面末尾的 `available:` 只列出这个 Attempt 实际可用的证据命令。
|
|
53
53
|
</Step>
|
|
54
54
|
<Step title="按问题读取证据">
|
|
55
55
|
```bash
|
|
@@ -59,7 +59,7 @@ AI 通常按任务选择这些入口:
|
|
|
59
59
|
|
|
60
60
|
`--execution` 合并 AI 输出与 trace:标准事件流提供消息、thinking、tool call/result 和 Skill load;OTel 在能够关联时给同一节点补开始时间、耗时、父子关系和错误状态。没有 OTel 时步骤仍完整,只不显示时间。
|
|
61
61
|
|
|
62
|
-
不带证据 flag 时,`show @<id>` 是失败诊断首页。它先列出失败断言的 group、matcher、expected、received、原因和源码位置,再给执行、生命周期阶段耗时与文件变化摘要。
|
|
62
|
+
不带证据 flag 时,`show @<id>` 是失败诊断首页。它先列出失败断言的 group、matcher、expected、received、原因和源码位置,再给执行、生命周期阶段耗时与文件变化摘要。Coding Agent 应先读取这一页;需要追查值的来源时,再打开对应证据。
|
|
63
63
|
|
|
64
64
|
```text
|
|
65
65
|
$ niceeval show @1k2m9qtr
|
|
@@ -96,7 +96,7 @@ AI 通常按任务选择这些入口:
|
|
|
96
96
|
|
|
97
97
|
full eval source: …/weather/brooklyn/a2/sources.json
|
|
98
98
|
available:
|
|
99
|
-
niceeval show @1k2m9qtr --
|
|
99
|
+
niceeval show @1k2m9qtr --source
|
|
100
100
|
niceeval show @1k2m9qtr --execution
|
|
101
101
|
niceeval show @1k2m9qtr --timing
|
|
102
102
|
```
|
|
@@ -153,7 +153,7 @@ AI 通常按任务选择这些入口:
|
|
|
153
153
|
full events: …/weather/brooklyn/a2/events.json
|
|
154
154
|
```
|
|
155
155
|
|
|
156
|
-
diff 是被测 Agent 在 Sandbox
|
|
156
|
+
diff 是被测 Agent 在 Sandbox 工作区造成的文件变化,不是评估用例源码的新旧差异。只有 Sandbox 评估用例才会收集到 diff——非 Sandbox 评估用例或 agent 确实没碰任何文件时,attempt 页的 `available:` 列表会省略 `--diff`。默认先给文件级摘要,避免把大段补丁塞进 Agent 上下文;`--diff=<文件>` 再展开单个文件,原始 artifact 路径始终保留:
|
|
157
157
|
|
|
158
158
|
```text
|
|
159
159
|
$ niceeval show @1c3h6tbn --diff
|
|
@@ -169,13 +169,13 @@ AI 通常按任务选择这些入口:
|
|
|
169
169
|
|
|
170
170
|
用这组输出检查信息是否足够:
|
|
171
171
|
|
|
172
|
-
|
|
|
172
|
+
| 诊断目标 | 入口 | 输出必须包含 |
|
|
173
173
|
| --- | --- | --- |
|
|
174
|
-
|
|
|
175
|
-
|
|
|
176
|
-
|
|
|
177
|
-
|
|
|
178
|
-
| Sandbox
|
|
174
|
+
| 查看一次 Attempt 摘要 | `niceeval show @<id>` | 评估用例断言、执行步骤、生命周期阶段耗时摘要、diff 摘要及各块可用性 |
|
|
175
|
+
| 检查评估用例断言及其通过或失败原因 | `niceeval show @<id> --source` | 运行时评估用例源码、源码哈希、断言所在行、严重度、分数与原因 |
|
|
176
|
+
| 检查 Coding Agent 的消息与调用 | `niceeval show @<id> --execution` | 消息、thinking、Skill load、工具调用与结果;有 OTel 时在同一节点附时间、父子关系和错误状态 |
|
|
177
|
+
| 检查 Attempt 各阶段耗时 | `niceeval show @<id> --timing` | lifecycle → hook/turn → shell → OTel 的统一时间树;出错的 Attempt 标出已知的最深失败节点 |
|
|
178
|
+
| 检查 Sandbox 工作区改动 | `niceeval show @<id> --diff` | 文件摘要、增删行数、具体补丁和原始 diff 路径;无文件工作区时明确 unavailable |
|
|
179
179
|
</Step>
|
|
180
180
|
<Step title="提出假设并修改">
|
|
181
181
|
根据证据只修改最可能出错的一侧:
|
|
@@ -184,9 +184,9 @@ AI 通常按任务选择这些入口:
|
|
|
184
184
|
| --- | --- |
|
|
185
185
|
| 回复或工具调用不符合需求 | 被测程序、Prompt、工具定义 |
|
|
186
186
|
| `diff` 与任务要求不符 | 被测 Agent 的实现策略或运行环境 |
|
|
187
|
-
| 正确行为被 gate 拒绝 |
|
|
187
|
+
| 正确行为被 gate 拒绝 | 评估用例断言、fixture、setup |
|
|
188
188
|
| timeout、鉴权或 Sandbox 错误 | 实验配置、Adapter、Sandbox |
|
|
189
|
-
|
|
|
189
|
+
| 同一评估用例时好时坏 | `--history`、`runs` 和稳定性问题 |
|
|
190
190
|
|
|
191
191
|
不要为了变绿而放宽一个本来正确的断言。先写清楚“哪条证据支持什么判断”,再修改代码。
|
|
192
192
|
</Step>
|
|
@@ -196,7 +196,7 @@ AI 通常按任务选择这些入口:
|
|
|
196
196
|
npx niceeval show weather/brooklyn
|
|
197
197
|
```
|
|
198
198
|
|
|
199
|
-
|
|
199
|
+
位置参数按评估用例 ID 前缀缩小实验范围。调试同一个失败时加 `--force`,确保刚才的修改真的触发一次新运行。然后用 `show` 验证判定、断言和证据是否按预期变化。
|
|
200
200
|
</Step>
|
|
201
201
|
<Step title="全量确认没有回归">
|
|
202
202
|
```bash
|
|
@@ -208,7 +208,7 @@ AI 通常按任务选择这些入口:
|
|
|
208
208
|
</Step>
|
|
209
209
|
</Steps>
|
|
210
210
|
|
|
211
|
-
##
|
|
211
|
+
## 从输出读取诊断信号
|
|
212
212
|
|
|
213
213
|
`--output agent` 运行中只向 stderr 追加低频 checkpoint(存活信号,不是结果数据源),结束时向 stdout 打印一个有界 handoff block——这才是 AI 应该解析的部分:
|
|
214
214
|
|
|
@@ -225,7 +225,7 @@ next:
|
|
|
225
225
|
niceeval show @1k2m9qtr --execution
|
|
226
226
|
```
|
|
227
227
|
|
|
228
|
-
|
|
228
|
+
Coding Agent 应先从 `failures` 选中 Attempt locator,再按证据位执行 `next` 给出的 `niceeval show @<id>` 或对应证据 flag。不要解析运行期间 stderr 上低频追加的 checkpoint 行;这些行只用于判断进程是否存活。失败条数超过上限(默认 5 条)时,handoff 只展开前几条并给出总数,完整清单从结果快照读取。机器读取以结果快照为事实来源:
|
|
229
229
|
|
|
230
230
|
```text
|
|
231
231
|
.niceeval/<experiment>/<快照>/
|
|
@@ -242,11 +242,11 @@ AI 应先从 `failures` 选中 Attempt locator,再按证据位执行 `next`
|
|
|
242
242
|
- `events.json` 是对话与工具调用事件,`trace.json` 是调用链,`diff.json` 是 Sandbox 文件变化。
|
|
243
243
|
- 某类证据不存在时,对应文件不会生成。先以 `show` 的提示为准,不要假设每个目录都有全部文件。
|
|
244
244
|
|
|
245
|
-
`show` 的默认结果可能合成自多次运行:每个
|
|
245
|
+
`show` 的默认结果可能合成自多次运行:每个 Experiment × 评估用例选择最新判定,因此局部重跑后仍能看到其它评估用例的旧结果。该视图用于查看每条评估用例的最新已知判定;同一版代码的整体结果必须通过一次 `--force` 全量运行确认。
|
|
246
246
|
|
|
247
|
-
##
|
|
247
|
+
## 结果复用条件
|
|
248
248
|
|
|
249
|
-
不传 `--force` 时,NiceEval
|
|
249
|
+
不传 `--force` 时,NiceEval 会比较当前指纹与最近结果。指纹由评估用例源码和运行配置组成,包括实验 ID、Agent、model、flags、Sandbox、timeout 与 strict 等设置。
|
|
250
250
|
|
|
251
251
|
| 最近结果与当前输入 | 本次行为 |
|
|
252
252
|
| --- | --- |
|
|
@@ -258,26 +258,26 @@ AI 应先从 `failures` 选中 Attempt locator,再按证据位执行 `next`
|
|
|
258
258
|
被测程序的源码不在指纹里。修改实现后,即使行为已经变化,旧的 `passed` 或 `failed` 仍可能被复用。因此可以这样选择:
|
|
259
259
|
|
|
260
260
|
- 只想重看已有结果:运行 `niceeval show`,不产生新费用。
|
|
261
|
-
-
|
|
262
|
-
-
|
|
263
|
-
- 准备结束本轮工作:对整个实验使用 `--force
|
|
261
|
+
- 修改了评估用例或实验配置:直接重跑;指纹变化会触发对应任务。
|
|
262
|
+
- 修改了被测程序:对受影响的评估用例使用 `--force`。
|
|
263
|
+
- 准备结束本轮工作:对整个实验使用 `--force`,排除其它评估用例的回归。
|
|
264
264
|
|
|
265
|
-
##
|
|
265
|
+
## 设置自主迭代协议
|
|
266
266
|
|
|
267
|
-
|
|
267
|
+
可以把下面的协议放进任务描述。它既适合优化被测实现,也适合调试评估用例:
|
|
268
268
|
|
|
269
269
|
```text
|
|
270
270
|
读取 node_modules/niceeval/INDEX.md,再按索引读取与任务有关的文档。
|
|
271
271
|
运行 npx niceeval exp local --output agent,并根据退出码和失败 locator 决定下一步。
|
|
272
272
|
对每个失败的 Eval,从报告选择一个 Attempt locator;再运行 niceeval show @<id>
|
|
273
|
-
并按问题选择 --
|
|
273
|
+
并按问题选择 --source、--execution、--timing 或 --diff;--timing 从 lifecycle 展开 setup/teardown
|
|
274
274
|
hook、shell 命令、每轮 send 与可关联的 OTel model/tool,回答整个 Attempt 的时间花在哪里。
|
|
275
275
|
写出失败原因的假设,并判断应该修改被测程序、
|
|
276
276
|
Eval,还是实验环境。修改后用 --force 重跑对应 Eval,比较新的判定和证据。
|
|
277
|
-
|
|
277
|
+
同一问题连续三轮没有新证据或改善时停止并汇报,不要为得到通过结果而放宽正确的断言。
|
|
278
278
|
全部局部失败清零后,用 npx niceeval exp local --output agent --force 全量验证;退出码 0 才完成。
|
|
279
279
|
```
|
|
280
280
|
|
|
281
281
|
真实 Agent 的运行可能产生费用。实验阶段可以加 `--budget <美元>` 限制本轮累计成本;预算只能限制单次命令,不能替代上面的停止条件。
|
|
282
282
|
|
|
283
|
-
人与 AI 随时可以接手同一轮工作。AI 用 `niceeval show` 读取的结果,也能由人运行 `npx niceeval view` 在网页中查看。两者读取同一批 artifact;完整的输出格式、历史选择和网页操作见[查看结果](/zh/
|
|
283
|
+
人与 AI 随时可以接手同一轮工作。AI 用 `niceeval show` 读取的结果,也能由人运行 `npx niceeval view` 在网页中查看。两者读取同一批 artifact;完整的输出格式、历史选择和网页操作见[查看结果](/zh/tutorials/viewing-results)。
|