@rpamis/comet 0.4.0-beta.2 → 0.4.0-beta.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +361 -136
- package/assets/eval-reports/comet-native-vs-beta16-beta17-20260810/native-benchmark-report.html +8 -0
- package/assets/eval-reports/comet-native-vs-beta16-beta17-20260810/native-benchmark-report.json +286 -0
- package/assets/manifest.json +53 -16
- package/assets/skills/comet/SKILL.md +16 -256
- package/assets/skills/comet/agents/openai.yaml +3 -0
- package/assets/skills/comet/rules/comet-phase-guard.en.md +39 -35
- package/assets/skills/comet/rules/comet-phase-guard.md +38 -35
- package/assets/skills/comet/rules/comet-workflow-guard.en.md +33 -0
- package/assets/skills/comet/rules/comet-workflow-guard.md +33 -0
- package/assets/skills/comet/scripts/comet-archive.mjs +165 -2
- package/assets/skills/comet/scripts/comet-entry-runtime.mjs +149 -0
- package/assets/skills/comet/scripts/comet-guard.mjs +207 -2
- package/assets/skills/comet/scripts/comet-handoff.mjs +165 -2
- package/assets/skills/comet/scripts/comet-hook-guard.mjs +153 -2
- package/assets/skills/comet/scripts/comet-hook-router.mjs +168 -0
- package/assets/skills/comet/scripts/comet-intent.mjs +10 -2
- package/assets/skills/comet/scripts/comet-resume-probe.mjs +149 -0
- package/assets/skills/comet/scripts/comet-runtime.mjs +293 -12935
- package/assets/skills/comet/scripts/comet-state.mjs +165 -2
- package/assets/skills/comet/scripts/comet-yaml-validate.mjs +149 -2
- package/assets/skills/comet-any/SKILL.md +7 -6
- package/assets/skills/comet-any/agents/openai.yaml +5 -0
- package/assets/skills/comet-any/reference/authored-zone-example.md +10 -5
- package/assets/skills/comet-any/reference/authoring-subagents.md +4 -12
- package/assets/skills/comet-any/reference/bundle-authoring.md +2 -2
- package/assets/skills/comet-any/reference/subagents/pause-points-author.md +15 -15
- package/assets/skills/comet-any/reference/subagents/reference-author.md +2 -5
- package/assets/skills/comet-any/reference/subagents/script-author.md +2 -5
- package/assets/skills/comet-any/reference/subagents/skill-core-author.md +2 -5
- package/assets/skills/comet-any/reference/subagents/skill-reviewer.md +5 -6
- package/assets/skills/comet-any/reference/subagents/workflow-entry-author.md +9 -11
- package/assets/skills/comet-archive/SKILL.md +69 -26
- package/assets/skills/comet-archive/agents/openai.yaml +5 -0
- package/assets/skills/comet-build/SKILL.md +94 -102
- package/assets/skills/comet-build/agents/openai.yaml +5 -0
- package/assets/skills/comet-classic/SKILL.md +301 -0
- package/assets/skills/comet-classic/agents/openai.yaml +3 -0
- package/assets/skills/comet-classic/reference/auto-transition.md +27 -0
- package/assets/skills/comet-classic/reference/classic-layout.md +34 -0
- package/assets/skills/comet-classic/reference/comet-yaml-fields.md +81 -0
- package/assets/skills/comet-classic/reference/context-recovery.md +55 -0
- package/assets/skills/{comet → comet-classic}/reference/debug-gate.md +2 -2
- package/assets/skills/comet-classic/reference/decision-point.md +41 -0
- package/assets/skills/comet-classic/reference/dirty-worktree.md +59 -0
- package/assets/skills/comet-classic/reference/file-structure.md +28 -0
- package/assets/skills/{comet → comet-classic}/reference/intent-frame.md +4 -4
- package/assets/skills/comet-classic/reference/scripts.md +73 -0
- package/assets/skills/{comet → comet-classic}/reference/subagent-dispatch.md +13 -14
- package/assets/skills/comet-classic/reference/workspace.md +23 -0
- package/assets/skills/comet-design/SKILL.md +46 -42
- package/assets/skills/comet-design/agents/openai.yaml +5 -0
- package/assets/skills/comet-hotfix/SKILL.md +62 -40
- package/assets/skills/comet-hotfix/agents/openai.yaml +5 -0
- package/assets/skills/comet-native/SKILL.md +108 -0
- package/assets/skills/comet-native/agents/openai.yaml +3 -0
- package/assets/skills/comet-native/reference/artifacts.md +71 -0
- package/assets/skills/comet-native/reference/clarification.md +79 -0
- package/assets/skills/comet-native/reference/commands.md +46 -0
- package/assets/skills/comet-native/reference/recovery.md +67 -0
- package/assets/skills/comet-native/reference/workspace.md +23 -0
- package/assets/skills/comet-native/scripts/comet-native-archive.mjs +217 -0
- package/assets/skills/comet-native/scripts/comet-native-doctor.mjs +263 -0
- package/assets/skills/comet-native/scripts/comet-native-hook-guard.mjs +167 -0
- package/assets/skills/comet-native/scripts/comet-native-init.mjs +181 -0
- package/assets/skills/comet-native/scripts/comet-native-new.mjs +207 -0
- package/assets/skills/comet-native/scripts/comet-native-next.mjs +213 -0
- package/assets/skills/comet-native/scripts/comet-native-root.mjs +190 -0
- package/assets/skills/comet-native/scripts/comet-native-runtime.mjs +299 -0
- package/assets/skills/comet-native/scripts/comet-native-select.mjs +185 -0
- package/assets/skills/comet-native/scripts/comet-native-show.mjs +158 -0
- package/assets/skills/comet-native/scripts/comet-native-spec.mjs +163 -0
- package/assets/skills/comet-native/scripts/comet-native-status.mjs +182 -0
- package/assets/skills/comet-open/SKILL.md +140 -61
- package/assets/skills/comet-open/agents/openai.yaml +5 -0
- package/assets/skills/comet-review/SKILL.md +121 -0
- package/assets/skills/comet-review/agents/openai.yaml +5 -0
- package/assets/skills/comet-tweak/SKILL.md +57 -32
- package/assets/skills/comet-tweak/agents/openai.yaml +5 -0
- package/assets/skills/comet-verify/SKILL.md +69 -77
- package/assets/skills/comet-verify/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet/SKILL.md +15 -244
- package/assets/skills-zh/comet/agents/openai.yaml +3 -0
- package/assets/skills-zh/comet-any/SKILL.md +7 -6
- package/assets/skills-zh/comet-any/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-any/reference/authored-zone-example.md +10 -5
- package/assets/skills-zh/comet-any/reference/authoring-subagents.md +4 -15
- package/assets/skills-zh/comet-any/reference/bundle-authoring.md +2 -2
- package/assets/skills-zh/comet-any/reference/subagents/pause-points-author.md +15 -16
- package/assets/skills-zh/comet-any/reference/subagents/reference-author.md +2 -5
- package/assets/skills-zh/comet-any/reference/subagents/script-author.md +2 -5
- package/assets/skills-zh/comet-any/reference/subagents/skill-core-author.md +2 -5
- package/assets/skills-zh/comet-any/reference/subagents/skill-reviewer.md +5 -6
- package/assets/skills-zh/comet-any/reference/subagents/workflow-entry-author.md +9 -11
- package/assets/skills-zh/comet-archive/SKILL.md +69 -26
- package/assets/skills-zh/comet-archive/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-build/SKILL.md +94 -102
- package/assets/skills-zh/comet-build/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-classic/SKILL.md +289 -0
- package/assets/skills-zh/comet-classic/agents/openai.yaml +3 -0
- package/assets/skills-zh/comet-classic/reference/auto-transition.md +27 -0
- package/assets/skills-zh/comet-classic/reference/classic-layout.md +34 -0
- package/assets/skills-zh/comet-classic/reference/comet-yaml-fields.md +80 -0
- package/assets/skills-zh/comet-classic/reference/context-recovery.md +55 -0
- package/assets/skills-zh/comet-classic/reference/debug-gate.md +33 -0
- package/assets/skills-zh/comet-classic/reference/decision-point.md +41 -0
- package/assets/skills-zh/comet-classic/reference/dirty-worktree.md +59 -0
- package/assets/skills-zh/comet-classic/reference/file-structure.md +28 -0
- package/assets/skills-zh/comet-classic/reference/intent-frame.md +74 -0
- package/assets/skills-zh/comet-classic/reference/scripts.md +73 -0
- package/assets/skills-zh/comet-classic/reference/subagent-dispatch.md +168 -0
- package/assets/skills-zh/comet-classic/reference/workspace.md +23 -0
- package/assets/skills-zh/comet-design/SKILL.md +46 -42
- package/assets/skills-zh/comet-design/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-hotfix/SKILL.md +61 -39
- package/assets/skills-zh/comet-hotfix/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-native/SKILL.md +108 -0
- package/assets/skills-zh/comet-native/agents/openai.yaml +3 -0
- package/assets/skills-zh/comet-native/reference/artifacts.md +71 -0
- package/assets/skills-zh/comet-native/reference/clarification.md +79 -0
- package/assets/skills-zh/comet-native/reference/commands.md +46 -0
- package/assets/skills-zh/comet-native/reference/recovery.md +67 -0
- package/assets/skills-zh/comet-native/reference/workspace.md +23 -0
- package/assets/skills-zh/comet-open/SKILL.md +139 -60
- package/assets/skills-zh/comet-open/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-review/SKILL.md +121 -0
- package/assets/skills-zh/comet-review/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-tweak/SKILL.md +56 -31
- package/assets/skills-zh/comet-tweak/agents/openai.yaml +5 -0
- package/assets/skills-zh/comet-verify/SKILL.md +69 -77
- package/assets/skills-zh/comet-verify/agents/openai.yaml +5 -0
- package/bin/comet.js +5 -1
- package/bin/fast-runtime-router.js +79 -0
- package/dist/app/cli/comet-banner.d.ts +25 -0
- package/dist/app/cli/comet-banner.d.ts.map +1 -0
- package/dist/app/cli/comet-banner.js +265 -0
- package/dist/app/cli/comet-banner.js.map +1 -0
- package/dist/app/cli/index.js +184 -28
- package/dist/app/cli/index.js.map +1 -1
- package/dist/app/commands/classic.d.ts +5 -0
- package/dist/app/commands/classic.d.ts.map +1 -0
- package/dist/app/commands/classic.js +34 -0
- package/dist/app/commands/classic.js.map +1 -0
- package/dist/app/commands/command-result.d.ts +6 -0
- package/dist/app/commands/command-result.d.ts.map +1 -0
- package/dist/app/commands/command-result.js +4 -0
- package/dist/app/commands/command-result.js.map +1 -0
- package/dist/app/commands/dashboard.d.ts.map +1 -1
- package/dist/app/commands/dashboard.js +2 -1
- package/dist/app/commands/dashboard.js.map +1 -1
- package/dist/app/commands/doctor.d.ts +4 -0
- package/dist/app/commands/doctor.d.ts.map +1 -1
- package/dist/app/commands/doctor.js +1066 -95
- package/dist/app/commands/doctor.js.map +1 -1
- package/dist/app/commands/eval.d.ts +10 -1
- package/dist/app/commands/eval.d.ts.map +1 -1
- package/dist/app/commands/eval.js +354 -51
- package/dist/app/commands/eval.js.map +1 -1
- package/dist/app/commands/i18n.d.ts +1 -1
- package/dist/app/commands/i18n.d.ts.map +1 -1
- package/dist/app/commands/i18n.js +118 -6
- package/dist/app/commands/i18n.js.map +1 -1
- package/dist/app/commands/init.d.ts +13 -3
- package/dist/app/commands/init.d.ts.map +1 -1
- package/dist/app/commands/init.js +627 -82
- package/dist/app/commands/init.js.map +1 -1
- package/dist/app/commands/native.d.ts +2 -0
- package/dist/app/commands/native.d.ts.map +1 -0
- package/dist/app/commands/native.js +10 -0
- package/dist/app/commands/native.js.map +1 -0
- package/dist/app/commands/project-scope-selection.d.ts +14 -0
- package/dist/app/commands/project-scope-selection.d.ts.map +1 -0
- package/dist/app/commands/project-scope-selection.js +33 -0
- package/dist/app/commands/project-scope-selection.js.map +1 -0
- package/dist/app/commands/resume-probe.d.ts +12 -0
- package/dist/app/commands/resume-probe.d.ts.map +1 -0
- package/dist/app/commands/resume-probe.js +68 -0
- package/dist/app/commands/resume-probe.js.map +1 -0
- package/dist/app/commands/status.d.ts.map +1 -1
- package/dist/app/commands/status.js +70 -104
- package/dist/app/commands/status.js.map +1 -1
- package/dist/app/commands/uninstall.d.ts +11 -0
- package/dist/app/commands/uninstall.d.ts.map +1 -1
- package/dist/app/commands/uninstall.js +526 -86
- package/dist/app/commands/uninstall.js.map +1 -1
- package/dist/app/commands/update.d.ts +30 -5
- package/dist/app/commands/update.d.ts.map +1 -1
- package/dist/app/commands/update.js +1641 -123
- package/dist/app/commands/update.js.map +1 -1
- package/dist/app/commands/workflow.d.ts +7 -0
- package/dist/app/commands/workflow.d.ts.map +1 -0
- package/dist/app/commands/workflow.js +18 -0
- package/dist/app/commands/workflow.js.map +1 -0
- package/dist/config/repository-layout.json +48 -3
- package/dist/domains/bundle/bundle-platform.d.ts +1 -0
- package/dist/domains/bundle/bundle-platform.d.ts.map +1 -1
- package/dist/domains/bundle/bundle-platform.js +18 -4
- package/dist/domains/bundle/bundle-platform.js.map +1 -1
- package/dist/domains/bundle/eval-manifest-runtime.d.ts +8 -0
- package/dist/domains/bundle/eval-manifest-runtime.d.ts.map +1 -0
- package/dist/domains/bundle/eval-manifest-runtime.js +118 -0
- package/dist/domains/bundle/eval-manifest-runtime.js.map +1 -0
- package/dist/domains/bundle/eval-run-result.d.ts +22 -0
- package/dist/domains/bundle/eval-run-result.d.ts.map +1 -0
- package/dist/domains/bundle/eval-run-result.js +207 -0
- package/dist/domains/bundle/eval-run-result.js.map +1 -0
- package/dist/domains/bundle/eval.d.ts.map +1 -1
- package/dist/domains/bundle/eval.js +21 -24
- package/dist/domains/bundle/eval.js.map +1 -1
- package/dist/domains/bundle/factory.d.ts.map +1 -1
- package/dist/domains/bundle/factory.js +2 -9
- package/dist/domains/bundle/factory.js.map +1 -1
- package/dist/domains/bundle/review-summary.js +1 -1
- package/dist/domains/bundle/review-summary.js.map +1 -1
- package/dist/domains/bundle/state.d.ts.map +1 -1
- package/dist/domains/bundle/state.js +117 -2
- package/dist/domains/bundle/state.js.map +1 -1
- package/dist/domains/bundle/types.d.ts +1 -1
- package/dist/domains/bundle/types.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-archive-entry.js +1 -1
- package/dist/domains/comet-classic/classic-archive-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-archive-pointer.d.ts +8 -0
- package/dist/domains/comet-classic/classic-archive-pointer.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-archive-pointer.js +66 -0
- package/dist/domains/comet-classic/classic-archive-pointer.js.map +1 -0
- package/dist/domains/comet-classic/classic-archive.d.ts +1 -0
- package/dist/domains/comet-classic/classic-archive.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-archive.js +205 -114
- package/dist/domains/comet-classic/classic-archive.js.map +1 -1
- package/dist/domains/comet-classic/classic-branch-binding.d.ts +51 -0
- package/dist/domains/comet-classic/classic-branch-binding.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-branch-binding.js +113 -0
- package/dist/domains/comet-classic/classic-branch-binding.js.map +1 -0
- package/dist/domains/comet-classic/classic-cli.d.ts +3 -1
- package/dist/domains/comet-classic/classic-cli.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-cli.js +15 -3
- package/dist/domains/comet-classic/classic-cli.js.map +1 -1
- package/dist/domains/comet-classic/classic-command-checks.d.ts +20 -0
- package/dist/domains/comet-classic/classic-command-checks.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-command-checks.js +94 -0
- package/dist/domains/comet-classic/classic-command-checks.js.map +1 -0
- package/dist/domains/comet-classic/classic-command-context.d.ts +21 -0
- package/dist/domains/comet-classic/classic-command-context.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-command-context.js +52 -0
- package/dist/domains/comet-classic/classic-command-context.js.map +1 -0
- package/dist/domains/comet-classic/classic-current-change.d.ts +17 -0
- package/dist/domains/comet-classic/classic-current-change.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-current-change.js +143 -0
- package/dist/domains/comet-classic/classic-current-change.js.map +1 -0
- package/dist/domains/comet-classic/classic-diagnostics.d.ts +2 -1
- package/dist/domains/comet-classic/classic-diagnostics.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-diagnostics.js +54 -1
- package/dist/domains/comet-classic/classic-diagnostics.js.map +1 -1
- package/dist/domains/comet-classic/classic-evidence.d.ts +1 -0
- package/dist/domains/comet-classic/classic-evidence.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-evidence.js +140 -40
- package/dist/domains/comet-classic/classic-evidence.js.map +1 -1
- package/dist/domains/comet-classic/classic-guard-entry.js +1 -1
- package/dist/domains/comet-classic/classic-guard-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-guard.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-guard.js +230 -209
- package/dist/domains/comet-classic/classic-guard.js.map +1 -1
- package/dist/domains/comet-classic/classic-handoff-entry.js +1 -1
- package/dist/domains/comet-classic/classic-handoff-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-handoff.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-handoff.js +195 -109
- package/dist/domains/comet-classic/classic-handoff.js.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard-entry.js +1 -1
- package/dist/domains/comet-classic/classic-hook-guard-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard.d.ts +13 -0
- package/dist/domains/comet-classic/classic-hook-guard.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard.js +448 -75
- package/dist/domains/comet-classic/classic-hook-guard.js.map +1 -1
- package/dist/domains/comet-classic/classic-intent-command.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-intent-command.js +2 -1
- package/dist/domains/comet-classic/classic-intent-command.js.map +1 -1
- package/dist/domains/comet-classic/classic-intent-entry.js +1 -1
- package/dist/domains/comet-classic/classic-intent-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-layout-initialization.d.ts +87 -0
- package/dist/domains/comet-classic/classic-layout-initialization.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-layout-initialization.js +828 -0
- package/dist/domains/comet-classic/classic-layout-initialization.js.map +1 -0
- package/dist/domains/comet-classic/classic-layout.d.ts +41 -0
- package/dist/domains/comet-classic/classic-layout.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-layout.js +222 -0
- package/dist/domains/comet-classic/classic-layout.js.map +1 -0
- package/dist/domains/comet-classic/classic-migrate.d.ts +4 -0
- package/dist/domains/comet-classic/classic-migrate.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-migrate.js +8 -12
- package/dist/domains/comet-classic/classic-migrate.js.map +1 -1
- package/dist/domains/comet-classic/classic-openspec-command.d.ts +4 -0
- package/dist/domains/comet-classic/classic-openspec-command.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-openspec-command.js +45 -0
- package/dist/domains/comet-classic/classic-openspec-command.js.map +1 -0
- package/dist/domains/comet-classic/classic-openspec-root.d.ts +13 -0
- package/dist/domains/comet-classic/classic-openspec-root.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-openspec-root.js +47 -0
- package/dist/domains/comet-classic/classic-openspec-root.js.map +1 -0
- package/dist/domains/comet-classic/classic-paths.d.ts +11 -1
- package/dist/domains/comet-classic/classic-paths.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-paths.js +87 -35
- package/dist/domains/comet-classic/classic-paths.js.map +1 -1
- package/dist/domains/comet-classic/classic-plan-readiness.d.ts +12 -0
- package/dist/domains/comet-classic/classic-plan-readiness.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-plan-readiness.js +26 -0
- package/dist/domains/comet-classic/classic-plan-readiness.js.map +1 -0
- package/dist/domains/comet-classic/classic-project-config.d.ts +17 -0
- package/dist/domains/comet-classic/classic-project-config.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-project-config.js +42 -0
- package/dist/domains/comet-classic/classic-project-config.js.map +1 -0
- package/dist/domains/comet-classic/classic-protected-path.d.ts +39 -0
- package/dist/domains/comet-classic/classic-protected-path.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-protected-path.js +69 -0
- package/dist/domains/comet-classic/classic-protected-path.js.map +1 -0
- package/dist/domains/comet-classic/classic-resolver.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-resolver.js +4 -4
- package/dist/domains/comet-classic/classic-resolver.js.map +1 -1
- package/dist/domains/comet-classic/classic-resume-probe-command.d.ts +3 -0
- package/dist/domains/comet-classic/classic-resume-probe-command.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe-command.js +60 -0
- package/dist/domains/comet-classic/classic-resume-probe-command.js.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.d.ts +2 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.js +4 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.js.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe.d.ts +29 -0
- package/dist/domains/comet-classic/classic-resume-probe.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe.js +406 -0
- package/dist/domains/comet-classic/classic-resume-probe.js.map +1 -0
- package/dist/domains/comet-classic/classic-root-command.d.ts +8 -0
- package/dist/domains/comet-classic/classic-root-command.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-root-command.js +199 -0
- package/dist/domains/comet-classic/classic-root-command.js.map +1 -0
- package/dist/domains/comet-classic/classic-root-move.d.ts +118 -0
- package/dist/domains/comet-classic/classic-root-move.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-root-move.js +1640 -0
- package/dist/domains/comet-classic/classic-root-move.js.map +1 -0
- package/dist/domains/comet-classic/classic-runtime-run.d.ts +8 -1
- package/dist/domains/comet-classic/classic-runtime-run.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-runtime-run.js +327 -5
- package/dist/domains/comet-classic/classic-runtime-run.js.map +1 -1
- package/dist/domains/comet-classic/classic-script-entry.d.ts +2 -2
- package/dist/domains/comet-classic/classic-script-entry.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-script-entry.js +4 -3
- package/dist/domains/comet-classic/classic-script-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-state-command.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-state-command.js +431 -99
- package/dist/domains/comet-classic/classic-state-command.js.map +1 -1
- package/dist/domains/comet-classic/classic-state-entry.js +1 -1
- package/dist/domains/comet-classic/classic-state-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-state-events.d.ts +1 -1
- package/dist/domains/comet-classic/classic-state-events.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-state-events.js +3 -4
- package/dist/domains/comet-classic/classic-state-events.js.map +1 -1
- package/dist/domains/comet-classic/classic-state.d.ts +6 -4
- package/dist/domains/comet-classic/classic-state.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-state.js +20 -7
- package/dist/domains/comet-classic/classic-state.js.map +1 -1
- package/dist/domains/comet-classic/classic-store.d.ts +6 -1
- package/dist/domains/comet-classic/classic-store.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-store.js +36 -23
- package/dist/domains/comet-classic/classic-store.js.map +1 -1
- package/dist/domains/comet-classic/classic-transitions.d.ts +1 -1
- package/dist/domains/comet-classic/classic-transitions.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-transitions.js +37 -3
- package/dist/domains/comet-classic/classic-transitions.js.map +1 -1
- package/dist/domains/comet-classic/classic-validate-command.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-validate-command.js +40 -20
- package/dist/domains/comet-classic/classic-validate-command.js.map +1 -1
- package/dist/domains/comet-classic/classic-validate-entry.js +1 -1
- package/dist/domains/comet-classic/classic-validate-entry.js.map +1 -1
- package/dist/domains/comet-classic/classic-workspace-command.d.ts +3 -0
- package/dist/domains/comet-classic/classic-workspace-command.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-workspace-command.js +45 -0
- package/dist/domains/comet-classic/classic-workspace-command.js.map +1 -0
- package/dist/domains/comet-classic/classic-workspace.d.ts +38 -0
- package/dist/domains/comet-classic/classic-workspace.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-workspace.js +315 -0
- package/dist/domains/comet-classic/classic-workspace.js.map +1 -0
- package/dist/domains/comet-classic/index.d.ts +3 -0
- package/dist/domains/comet-classic/index.d.ts.map +1 -1
- package/dist/domains/comet-classic/index.js +3 -0
- package/dist/domains/comet-classic/index.js.map +1 -1
- package/dist/domains/comet-entry/current-selection-repair.d.ts +17 -0
- package/dist/domains/comet-entry/current-selection-repair.d.ts.map +1 -0
- package/dist/domains/comet-entry/current-selection-repair.js +19 -0
- package/dist/domains/comet-entry/current-selection-repair.js.map +1 -0
- package/dist/domains/comet-entry/current-selection.d.ts +28 -0
- package/dist/domains/comet-entry/current-selection.d.ts.map +1 -0
- package/dist/domains/comet-entry/current-selection.js +114 -0
- package/dist/domains/comet-entry/current-selection.js.map +1 -0
- package/dist/domains/comet-entry/entry-reads.d.ts +17 -0
- package/dist/domains/comet-entry/entry-reads.d.ts.map +1 -0
- package/dist/domains/comet-entry/entry-reads.js +20 -0
- package/dist/domains/comet-entry/entry-reads.js.map +1 -0
- package/dist/domains/comet-entry/entry-runtime-entry.d.ts +2 -0
- package/dist/domains/comet-entry/entry-runtime-entry.d.ts.map +1 -0
- package/dist/domains/comet-entry/entry-runtime-entry.js +3 -0
- package/dist/domains/comet-entry/entry-runtime-entry.js.map +1 -0
- package/dist/domains/comet-entry/entry-runtime.d.ts +7 -0
- package/dist/domains/comet-entry/entry-runtime.d.ts.map +1 -0
- package/dist/domains/comet-entry/entry-runtime.js +55 -0
- package/dist/domains/comet-entry/entry-runtime.js.map +1 -0
- package/dist/domains/comet-entry/hook-adapter.d.ts +6 -0
- package/dist/domains/comet-entry/hook-adapter.d.ts.map +1 -0
- package/dist/domains/comet-entry/hook-adapter.js +225 -0
- package/dist/domains/comet-entry/hook-adapter.js.map +1 -0
- package/dist/domains/comet-entry/hook-project-root.d.ts +3 -0
- package/dist/domains/comet-entry/hook-project-root.d.ts.map +1 -0
- package/dist/domains/comet-entry/hook-project-root.js +62 -0
- package/dist/domains/comet-entry/hook-project-root.js.map +1 -0
- package/dist/domains/comet-entry/hook-router-entry.d.ts +11 -0
- package/dist/domains/comet-entry/hook-router-entry.d.ts.map +1 -0
- package/dist/domains/comet-entry/hook-router-entry.js +121 -0
- package/dist/domains/comet-entry/hook-router-entry.js.map +1 -0
- package/dist/domains/comet-entry/hook-router.d.ts +44 -0
- package/dist/domains/comet-entry/hook-router.d.ts.map +1 -0
- package/dist/domains/comet-entry/hook-router.js +200 -0
- package/dist/domains/comet-entry/hook-router.js.map +1 -0
- package/dist/domains/comet-entry/hook-types.d.ts +21 -0
- package/dist/domains/comet-entry/hook-types.d.ts.map +1 -0
- package/dist/domains/comet-entry/hook-types.js +2 -0
- package/dist/domains/comet-entry/hook-types.js.map +1 -0
- package/dist/domains/comet-entry/init-workflow.d.ts +19 -0
- package/dist/domains/comet-entry/init-workflow.d.ts.map +1 -0
- package/dist/domains/comet-entry/init-workflow.js +138 -0
- package/dist/domains/comet-entry/init-workflow.js.map +1 -0
- package/dist/domains/comet-entry/project-activation.d.ts +12 -0
- package/dist/domains/comet-entry/project-activation.d.ts.map +1 -0
- package/dist/domains/comet-entry/project-activation.js +110 -0
- package/dist/domains/comet-entry/project-activation.js.map +1 -0
- package/dist/domains/comet-entry/project-status.d.ts +3 -0
- package/dist/domains/comet-entry/project-status.d.ts.map +1 -0
- package/dist/domains/comet-entry/project-status.js +291 -0
- package/dist/domains/comet-entry/project-status.js.map +1 -0
- package/dist/domains/comet-entry/resolve-entry.d.ts +3 -0
- package/dist/domains/comet-entry/resolve-entry.d.ts.map +1 -0
- package/dist/domains/comet-entry/resolve-entry.js +17 -0
- package/dist/domains/comet-entry/resolve-entry.js.map +1 -0
- package/dist/domains/comet-entry/resume-probe.d.ts +34 -0
- package/dist/domains/comet-entry/resume-probe.d.ts.map +1 -0
- package/dist/domains/comet-entry/resume-probe.js +538 -0
- package/dist/domains/comet-entry/resume-probe.js.map +1 -0
- package/dist/domains/comet-entry/types.d.ts +60 -0
- package/dist/domains/comet-entry/types.d.ts.map +1 -0
- package/dist/domains/comet-entry/types.js +2 -0
- package/dist/domains/comet-entry/types.js.map +1 -0
- package/dist/domains/comet-entry/workflow-resolution.d.ts +8 -0
- package/dist/domains/comet-entry/workflow-resolution.d.ts.map +1 -0
- package/dist/domains/comet-entry/workflow-resolution.js +16 -0
- package/dist/domains/comet-entry/workflow-resolution.js.map +1 -0
- package/dist/domains/comet-native/index.d.ts +36 -0
- package/dist/domains/comet-native/index.d.ts.map +1 -0
- package/dist/domains/comet-native/index.js +36 -0
- package/dist/domains/comet-native/index.js.map +1 -0
- package/dist/domains/comet-native/native-acceptance.d.ts +53 -0
- package/dist/domains/comet-native/native-acceptance.d.ts.map +1 -0
- package/dist/domains/comet-native/native-acceptance.js +639 -0
- package/dist/domains/comet-native/native-acceptance.js.map +1 -0
- package/dist/domains/comet-native/native-archive-command.d.ts +3 -0
- package/dist/domains/comet-native/native-archive-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive-command.js +236 -0
- package/dist/domains/comet-native/native-archive-command.js.map +1 -0
- package/dist/domains/comet-native/native-archive-content.d.ts +26 -0
- package/dist/domains/comet-native/native-archive-content.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive-content.js +159 -0
- package/dist/domains/comet-native/native-archive-content.js.map +1 -0
- package/dist/domains/comet-native/native-archive-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-archive-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive-entry.js +4 -0
- package/dist/domains/comet-native/native-archive-entry.js.map +1 -0
- package/dist/domains/comet-native/native-archive-inspection.d.ts +9 -0
- package/dist/domains/comet-native/native-archive-inspection.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive-inspection.js +108 -0
- package/dist/domains/comet-native/native-archive-inspection.js.map +1 -0
- package/dist/domains/comet-native/native-archive-preflight.d.ts +64 -0
- package/dist/domains/comet-native/native-archive-preflight.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive-preflight.js +266 -0
- package/dist/domains/comet-native/native-archive-preflight.js.map +1 -0
- package/dist/domains/comet-native/native-archive-transaction.d.ts +20 -0
- package/dist/domains/comet-native/native-archive-transaction.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive-transaction.js +1188 -0
- package/dist/domains/comet-native/native-archive-transaction.js.map +1 -0
- package/dist/domains/comet-native/native-archive.d.ts +37 -0
- package/dist/domains/comet-native/native-archive.d.ts.map +1 -0
- package/dist/domains/comet-native/native-archive.js +466 -0
- package/dist/domains/comet-native/native-archive.js.map +1 -0
- package/dist/domains/comet-native/native-artifacts.d.ts +10 -0
- package/dist/domains/comet-native/native-artifacts.d.ts.map +1 -0
- package/dist/domains/comet-native/native-artifacts.js +204 -0
- package/dist/domains/comet-native/native-artifacts.js.map +1 -0
- package/dist/domains/comet-native/native-atomic-file.d.ts +10 -0
- package/dist/domains/comet-native/native-atomic-file.d.ts.map +1 -0
- package/dist/domains/comet-native/native-atomic-file.js +186 -0
- package/dist/domains/comet-native/native-atomic-file.js.map +1 -0
- package/dist/domains/comet-native/native-bounded-file.d.ts +49 -0
- package/dist/domains/comet-native/native-bounded-file.d.ts.map +1 -0
- package/dist/domains/comet-native/native-bounded-file.js +265 -0
- package/dist/domains/comet-native/native-bounded-file.js.map +1 -0
- package/dist/domains/comet-native/native-build-evidence.d.ts +42 -0
- package/dist/domains/comet-native/native-build-evidence.d.ts.map +1 -0
- package/dist/domains/comet-native/native-build-evidence.js +279 -0
- package/dist/domains/comet-native/native-build-evidence.js.map +1 -0
- package/dist/domains/comet-native/native-canonical-hash.d.ts +10 -0
- package/dist/domains/comet-native/native-canonical-hash.d.ts.map +1 -0
- package/dist/domains/comet-native/native-canonical-hash.js +100 -0
- package/dist/domains/comet-native/native-canonical-hash.js.map +1 -0
- package/dist/domains/comet-native/native-change-recovery.d.ts +11 -0
- package/dist/domains/comet-native/native-change-recovery.d.ts.map +1 -0
- package/dist/domains/comet-native/native-change-recovery.js +15 -0
- package/dist/domains/comet-native/native-change-recovery.js.map +1 -0
- package/dist/domains/comet-native/native-change.d.ts +81 -0
- package/dist/domains/comet-native/native-change.d.ts.map +1 -0
- package/dist/domains/comet-native/native-change.js +895 -0
- package/dist/domains/comet-native/native-change.js.map +1 -0
- package/dist/domains/comet-native/native-check-command.d.ts +3 -0
- package/dist/domains/comet-native/native-check-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-check-command.js +30 -0
- package/dist/domains/comet-native/native-check-command.js.map +1 -0
- package/dist/domains/comet-native/native-check-executor.d.ts +36 -0
- package/dist/domains/comet-native/native-check-executor.d.ts.map +1 -0
- package/dist/domains/comet-native/native-check-executor.js +132 -0
- package/dist/domains/comet-native/native-check-executor.js.map +1 -0
- package/dist/domains/comet-native/native-check-receipt-model.d.ts +72 -0
- package/dist/domains/comet-native/native-check-receipt-model.d.ts.map +1 -0
- package/dist/domains/comet-native/native-check-receipt-model.js +414 -0
- package/dist/domains/comet-native/native-check-receipt-model.js.map +1 -0
- package/dist/domains/comet-native/native-check-receipt-storage.d.ts +10 -0
- package/dist/domains/comet-native/native-check-receipt-storage.d.ts.map +1 -0
- package/dist/domains/comet-native/native-check-receipt-storage.js +201 -0
- package/dist/domains/comet-native/native-check-receipt-storage.js.map +1 -0
- package/dist/domains/comet-native/native-check-receipt.d.ts +19 -0
- package/dist/domains/comet-native/native-check-receipt.d.ts.map +1 -0
- package/dist/domains/comet-native/native-check-receipt.js +516 -0
- package/dist/domains/comet-native/native-check-receipt.js.map +1 -0
- package/dist/domains/comet-native/native-check.d.ts +24 -0
- package/dist/domains/comet-native/native-check.d.ts.map +1 -0
- package/dist/domains/comet-native/native-check.js +50 -0
- package/dist/domains/comet-native/native-check.js.map +1 -0
- package/dist/domains/comet-native/native-checkpoint-command.d.ts +3 -0
- package/dist/domains/comet-native/native-checkpoint-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-checkpoint-command.js +33 -0
- package/dist/domains/comet-native/native-checkpoint-command.js.map +1 -0
- package/dist/domains/comet-native/native-checkpoint-journal.d.ts +13 -0
- package/dist/domains/comet-native/native-checkpoint-journal.d.ts.map +1 -0
- package/dist/domains/comet-native/native-checkpoint-journal.js +53 -0
- package/dist/domains/comet-native/native-checkpoint-journal.js.map +1 -0
- package/dist/domains/comet-native/native-checkpoint-storage.d.ts +43 -0
- package/dist/domains/comet-native/native-checkpoint-storage.d.ts.map +1 -0
- package/dist/domains/comet-native/native-checkpoint-storage.js +495 -0
- package/dist/domains/comet-native/native-checkpoint-storage.js.map +1 -0
- package/dist/domains/comet-native/native-children.d.ts +65 -0
- package/dist/domains/comet-native/native-children.d.ts.map +1 -0
- package/dist/domains/comet-native/native-children.js +453 -0
- package/dist/domains/comet-native/native-children.js.map +1 -0
- package/dist/domains/comet-native/native-cli-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-cli-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-cli-entry.js +18 -0
- package/dist/domains/comet-native/native-cli-entry.js.map +1 -0
- package/dist/domains/comet-native/native-cli-help.d.ts +6 -0
- package/dist/domains/comet-native/native-cli-help.d.ts.map +1 -0
- package/dist/domains/comet-native/native-cli-help.js +198 -0
- package/dist/domains/comet-native/native-cli-help.js.map +1 -0
- package/dist/domains/comet-native/native-cli-shared.d.ts +40 -0
- package/dist/domains/comet-native/native-cli-shared.d.ts.map +1 -0
- package/dist/domains/comet-native/native-cli-shared.js +328 -0
- package/dist/domains/comet-native/native-cli-shared.js.map +1 -0
- package/dist/domains/comet-native/native-cli.d.ts +4 -0
- package/dist/domains/comet-native/native-cli.d.ts.map +1 -0
- package/dist/domains/comet-native/native-cli.js +72 -0
- package/dist/domains/comet-native/native-cli.js.map +1 -0
- package/dist/domains/comet-native/native-config.d.ts +24 -0
- package/dist/domains/comet-native/native-config.d.ts.map +1 -0
- package/dist/domains/comet-native/native-config.js +61 -0
- package/dist/domains/comet-native/native-config.js.map +1 -0
- package/dist/domains/comet-native/native-conflict-inspection.d.ts +19 -0
- package/dist/domains/comet-native/native-conflict-inspection.d.ts.map +1 -0
- package/dist/domains/comet-native/native-conflict-inspection.js +100 -0
- package/dist/domains/comet-native/native-conflict-inspection.js.map +1 -0
- package/dist/domains/comet-native/native-conflict-radar.d.ts +75 -0
- package/dist/domains/comet-native/native-conflict-radar.d.ts.map +1 -0
- package/dist/domains/comet-native/native-conflict-radar.js +326 -0
- package/dist/domains/comet-native/native-conflict-radar.js.map +1 -0
- package/dist/domains/comet-native/native-continuation.d.ts +12 -0
- package/dist/domains/comet-native/native-continuation.d.ts.map +1 -0
- package/dist/domains/comet-native/native-continuation.js +274 -0
- package/dist/domains/comet-native/native-continuation.js.map +1 -0
- package/dist/domains/comet-native/native-contract-files.d.ts +24 -0
- package/dist/domains/comet-native/native-contract-files.d.ts.map +1 -0
- package/dist/domains/comet-native/native-contract-files.js +66 -0
- package/dist/domains/comet-native/native-contract-files.js.map +1 -0
- package/dist/domains/comet-native/native-contract.d.ts +41 -0
- package/dist/domains/comet-native/native-contract.d.ts.map +1 -0
- package/dist/domains/comet-native/native-contract.js +142 -0
- package/dist/domains/comet-native/native-contract.js.map +1 -0
- package/dist/domains/comet-native/native-diagnostics.d.ts +27 -0
- package/dist/domains/comet-native/native-diagnostics.d.ts.map +1 -0
- package/dist/domains/comet-native/native-diagnostics.js +645 -0
- package/dist/domains/comet-native/native-diagnostics.js.map +1 -0
- package/dist/domains/comet-native/native-doctor-command.d.ts +3 -0
- package/dist/domains/comet-native/native-doctor-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-doctor-command.js +361 -0
- package/dist/domains/comet-native/native-doctor-command.js.map +1 -0
- package/dist/domains/comet-native/native-doctor-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-doctor-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-doctor-entry.js +4 -0
- package/dist/domains/comet-native/native-doctor-entry.js.map +1 -0
- package/dist/domains/comet-native/native-doctor.d.ts +13 -0
- package/dist/domains/comet-native/native-doctor.d.ts.map +1 -0
- package/dist/domains/comet-native/native-doctor.js +865 -0
- package/dist/domains/comet-native/native-doctor.js.map +1 -0
- package/dist/domains/comet-native/native-evidence-command.d.ts +3 -0
- package/dist/domains/comet-native/native-evidence-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-evidence-command.js +35 -0
- package/dist/domains/comet-native/native-evidence-command.js.map +1 -0
- package/dist/domains/comet-native/native-evidence-projection.d.ts +50 -0
- package/dist/domains/comet-native/native-evidence-projection.d.ts.map +1 -0
- package/dist/domains/comet-native/native-evidence-projection.js +284 -0
- package/dist/domains/comet-native/native-evidence-projection.js.map +1 -0
- package/dist/domains/comet-native/native-evidence-retention.d.ts +37 -0
- package/dist/domains/comet-native/native-evidence-retention.d.ts.map +1 -0
- package/dist/domains/comet-native/native-evidence-retention.js +902 -0
- package/dist/domains/comet-native/native-evidence-retention.js.map +1 -0
- package/dist/domains/comet-native/native-evidence-storage.d.ts +65 -0
- package/dist/domains/comet-native/native-evidence-storage.d.ts.map +1 -0
- package/dist/domains/comet-native/native-evidence-storage.js +473 -0
- package/dist/domains/comet-native/native-evidence-storage.js.map +1 -0
- package/dist/domains/comet-native/native-file-identity.d.ts +3 -0
- package/dist/domains/comet-native/native-file-identity.d.ts.map +1 -0
- package/dist/domains/comet-native/native-file-identity.js +2 -0
- package/dist/domains/comet-native/native-file-identity.js.map +1 -0
- package/dist/domains/comet-native/native-findings.d.ts +8 -0
- package/dist/domains/comet-native/native-findings.d.ts.map +1 -0
- package/dist/domains/comet-native/native-findings.js +328 -0
- package/dist/domains/comet-native/native-findings.js.map +1 -0
- package/dist/domains/comet-native/native-git-provenance.d.ts +16 -0
- package/dist/domains/comet-native/native-git-provenance.d.ts.map +1 -0
- package/dist/domains/comet-native/native-git-provenance.js +171 -0
- package/dist/domains/comet-native/native-git-provenance.js.map +1 -0
- package/dist/domains/comet-native/native-guards.d.ts +8 -0
- package/dist/domains/comet-native/native-guards.d.ts.map +1 -0
- package/dist/domains/comet-native/native-guards.js +140 -0
- package/dist/domains/comet-native/native-guards.js.map +1 -0
- package/dist/domains/comet-native/native-hash.d.ts +3 -0
- package/dist/domains/comet-native/native-hash.d.ts.map +1 -0
- package/dist/domains/comet-native/native-hash.js +12 -0
- package/dist/domains/comet-native/native-hash.js.map +1 -0
- package/dist/domains/comet-native/native-hook-guard-command.d.ts +3 -0
- package/dist/domains/comet-native/native-hook-guard-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-hook-guard-command.js +32 -0
- package/dist/domains/comet-native/native-hook-guard-command.js.map +1 -0
- package/dist/domains/comet-native/native-hook-guard-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-hook-guard-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-hook-guard-entry.js +4 -0
- package/dist/domains/comet-native/native-hook-guard-entry.js.map +1 -0
- package/dist/domains/comet-native/native-hook-guard.d.ts +17 -0
- package/dist/domains/comet-native/native-hook-guard.d.ts.map +1 -0
- package/dist/domains/comet-native/native-hook-guard.js +353 -0
- package/dist/domains/comet-native/native-hook-guard.js.map +1 -0
- package/dist/domains/comet-native/native-init-command.d.ts +3 -0
- package/dist/domains/comet-native/native-init-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-init-command.js +41 -0
- package/dist/domains/comet-native/native-init-command.js.map +1 -0
- package/dist/domains/comet-native/native-init-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-init-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-init-entry.js +4 -0
- package/dist/domains/comet-native/native-init-entry.js.map +1 -0
- package/dist/domains/comet-native/native-local-execution.d.ts +25 -0
- package/dist/domains/comet-native/native-local-execution.d.ts.map +1 -0
- package/dist/domains/comet-native/native-local-execution.js +241 -0
- package/dist/domains/comet-native/native-local-execution.js.map +1 -0
- package/dist/domains/comet-native/native-lock.d.ts +47 -0
- package/dist/domains/comet-native/native-lock.d.ts.map +1 -0
- package/dist/domains/comet-native/native-lock.js +390 -0
- package/dist/domains/comet-native/native-lock.js.map +1 -0
- package/dist/domains/comet-native/native-loop-runtime.d.ts +63 -0
- package/dist/domains/comet-native/native-loop-runtime.d.ts.map +1 -0
- package/dist/domains/comet-native/native-loop-runtime.js +580 -0
- package/dist/domains/comet-native/native-loop-runtime.js.map +1 -0
- package/dist/domains/comet-native/native-mutation-lock.d.ts +7 -0
- package/dist/domains/comet-native/native-mutation-lock.d.ts.map +1 -0
- package/dist/domains/comet-native/native-mutation-lock.js +87 -0
- package/dist/domains/comet-native/native-mutation-lock.js.map +1 -0
- package/dist/domains/comet-native/native-new-command.d.ts +3 -0
- package/dist/domains/comet-native/native-new-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-new-command.js +57 -0
- package/dist/domains/comet-native/native-new-command.js.map +1 -0
- package/dist/domains/comet-native/native-new-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-new-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-new-entry.js +4 -0
- package/dist/domains/comet-native/native-new-entry.js.map +1 -0
- package/dist/domains/comet-native/native-next-command.d.ts +3 -0
- package/dist/domains/comet-native/native-next-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-next-command.js +292 -0
- package/dist/domains/comet-native/native-next-command.js.map +1 -0
- package/dist/domains/comet-native/native-next-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-next-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-next-entry.js +4 -0
- package/dist/domains/comet-native/native-next-entry.js.map +1 -0
- package/dist/domains/comet-native/native-paths.d.ts +28 -0
- package/dist/domains/comet-native/native-paths.d.ts.map +1 -0
- package/dist/domains/comet-native/native-paths.js +272 -0
- package/dist/domains/comet-native/native-paths.js.map +1 -0
- package/dist/domains/comet-native/native-portable-acceptance.d.ts +28 -0
- package/dist/domains/comet-native/native-portable-acceptance.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-acceptance.js +73 -0
- package/dist/domains/comet-native/native-portable-acceptance.js.map +1 -0
- package/dist/domains/comet-native/native-portable-archive.d.ts +37 -0
- package/dist/domains/comet-native/native-portable-archive.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-archive.js +492 -0
- package/dist/domains/comet-native/native-portable-archive.js.map +1 -0
- package/dist/domains/comet-native/native-portable-continuation.d.ts +42 -0
- package/dist/domains/comet-native/native-portable-continuation.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-continuation.js +469 -0
- package/dist/domains/comet-native/native-portable-continuation.js.map +1 -0
- package/dist/domains/comet-native/native-portable-migration-runtime.d.ts +9 -0
- package/dist/domains/comet-native/native-portable-migration-runtime.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-migration-runtime.js +218 -0
- package/dist/domains/comet-native/native-portable-migration-runtime.js.map +1 -0
- package/dist/domains/comet-native/native-portable-migration.d.ts +42 -0
- package/dist/domains/comet-native/native-portable-migration.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-migration.js +212 -0
- package/dist/domains/comet-native/native-portable-migration.js.map +1 -0
- package/dist/domains/comet-native/native-portable-recovery.d.ts +15 -0
- package/dist/domains/comet-native/native-portable-recovery.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-recovery.js +203 -0
- package/dist/domains/comet-native/native-portable-recovery.js.map +1 -0
- package/dist/domains/comet-native/native-portable-runtime.d.ts +168 -0
- package/dist/domains/comet-native/native-portable-runtime.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-runtime.js +1608 -0
- package/dist/domains/comet-native/native-portable-runtime.js.map +1 -0
- package/dist/domains/comet-native/native-portable-state.d.ts +30 -0
- package/dist/domains/comet-native/native-portable-state.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-state.js +701 -0
- package/dist/domains/comet-native/native-portable-state.js.map +1 -0
- package/dist/domains/comet-native/native-portable-status.d.ts +65 -0
- package/dist/domains/comet-native/native-portable-status.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-status.js +133 -0
- package/dist/domains/comet-native/native-portable-status.js.map +1 -0
- package/dist/domains/comet-native/native-portable-text.d.ts +8 -0
- package/dist/domains/comet-native/native-portable-text.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-text.js +28 -0
- package/dist/domains/comet-native/native-portable-text.js.map +1 -0
- package/dist/domains/comet-native/native-portable-transactions.d.ts +41 -0
- package/dist/domains/comet-native/native-portable-transactions.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-transactions.js +204 -0
- package/dist/domains/comet-native/native-portable-transactions.js.map +1 -0
- package/dist/domains/comet-native/native-portable-types.d.ts +166 -0
- package/dist/domains/comet-native/native-portable-types.d.ts.map +1 -0
- package/dist/domains/comet-native/native-portable-types.js +18 -0
- package/dist/domains/comet-native/native-portable-types.js.map +1 -0
- package/dist/domains/comet-native/native-progress-checkpoint.d.ts +13 -0
- package/dist/domains/comet-native/native-progress-checkpoint.d.ts.map +1 -0
- package/dist/domains/comet-native/native-progress-checkpoint.js +104 -0
- package/dist/domains/comet-native/native-progress-checkpoint.js.map +1 -0
- package/dist/domains/comet-native/native-protected-file.d.ts +113 -0
- package/dist/domains/comet-native/native-protected-file.d.ts.map +1 -0
- package/dist/domains/comet-native/native-protected-file.js +510 -0
- package/dist/domains/comet-native/native-protected-file.js.map +1 -0
- package/dist/domains/comet-native/native-pull-request-finish.d.ts +35 -0
- package/dist/domains/comet-native/native-pull-request-finish.d.ts.map +1 -0
- package/dist/domains/comet-native/native-pull-request-finish.js +282 -0
- package/dist/domains/comet-native/native-pull-request-finish.js.map +1 -0
- package/dist/domains/comet-native/native-receipt-command.d.ts +3 -0
- package/dist/domains/comet-native/native-receipt-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-receipt-command.js +120 -0
- package/dist/domains/comet-native/native-receipt-command.js.map +1 -0
- package/dist/domains/comet-native/native-receipt-errors.d.ts +23 -0
- package/dist/domains/comet-native/native-receipt-errors.d.ts.map +1 -0
- package/dist/domains/comet-native/native-receipt-errors.js +9 -0
- package/dist/domains/comet-native/native-receipt-errors.js.map +1 -0
- package/dist/domains/comet-native/native-receipt-refresh.d.ts +89 -0
- package/dist/domains/comet-native/native-receipt-refresh.d.ts.map +1 -0
- package/dist/domains/comet-native/native-receipt-refresh.js +243 -0
- package/dist/domains/comet-native/native-receipt-refresh.js.map +1 -0
- package/dist/domains/comet-native/native-redaction.d.ts +2 -0
- package/dist/domains/comet-native/native-redaction.d.ts.map +1 -0
- package/dist/domains/comet-native/native-redaction.js +18 -0
- package/dist/domains/comet-native/native-redaction.js.map +1 -0
- package/dist/domains/comet-native/native-repair-integration.d.ts +44 -0
- package/dist/domains/comet-native/native-repair-integration.d.ts.map +1 -0
- package/dist/domains/comet-native/native-repair-integration.js +232 -0
- package/dist/domains/comet-native/native-repair-integration.js.map +1 -0
- package/dist/domains/comet-native/native-repair-runtime.d.ts +77 -0
- package/dist/domains/comet-native/native-repair-runtime.d.ts.map +1 -0
- package/dist/domains/comet-native/native-repair-runtime.js +586 -0
- package/dist/domains/comet-native/native-repair-runtime.js.map +1 -0
- package/dist/domains/comet-native/native-repair-stagnation.d.ts +81 -0
- package/dist/domains/comet-native/native-repair-stagnation.d.ts.map +1 -0
- package/dist/domains/comet-native/native-repair-stagnation.js +298 -0
- package/dist/domains/comet-native/native-repair-stagnation.js.map +1 -0
- package/dist/domains/comet-native/native-resume-view.d.ts +14 -0
- package/dist/domains/comet-native/native-resume-view.d.ts.map +1 -0
- package/dist/domains/comet-native/native-resume-view.js +101 -0
- package/dist/domains/comet-native/native-resume-view.js.map +1 -0
- package/dist/domains/comet-native/native-revision.d.ts +20 -0
- package/dist/domains/comet-native/native-revision.d.ts.map +1 -0
- package/dist/domains/comet-native/native-revision.js +32 -0
- package/dist/domains/comet-native/native-revision.js.map +1 -0
- package/dist/domains/comet-native/native-root-command.d.ts +3 -0
- package/dist/domains/comet-native/native-root-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-root-command.js +29 -0
- package/dist/domains/comet-native/native-root-command.js.map +1 -0
- package/dist/domains/comet-native/native-root-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-root-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-root-entry.js +4 -0
- package/dist/domains/comet-native/native-root-entry.js.map +1 -0
- package/dist/domains/comet-native/native-root-move.d.ts +20 -0
- package/dist/domains/comet-native/native-root-move.d.ts.map +1 -0
- package/dist/domains/comet-native/native-root-move.js +855 -0
- package/dist/domains/comet-native/native-root-move.js.map +1 -0
- package/dist/domains/comet-native/native-run-consistency.d.ts +3 -0
- package/dist/domains/comet-native/native-run-consistency.d.ts.map +1 -0
- package/dist/domains/comet-native/native-run-consistency.js +132 -0
- package/dist/domains/comet-native/native-run-consistency.js.map +1 -0
- package/dist/domains/comet-native/native-run-store.d.ts +41 -0
- package/dist/domains/comet-native/native-run-store.d.ts.map +1 -0
- package/dist/domains/comet-native/native-run-store.js +452 -0
- package/dist/domains/comet-native/native-run-store.js.map +1 -0
- package/dist/domains/comet-native/native-runner-input.d.ts +160 -0
- package/dist/domains/comet-native/native-runner-input.d.ts.map +1 -0
- package/dist/domains/comet-native/native-runner-input.js +347 -0
- package/dist/domains/comet-native/native-runner-input.js.map +1 -0
- package/dist/domains/comet-native/native-runner-protocol.d.ts +40 -0
- package/dist/domains/comet-native/native-runner-protocol.d.ts.map +1 -0
- package/dist/domains/comet-native/native-runner-protocol.js +55 -0
- package/dist/domains/comet-native/native-runner-protocol.js.map +1 -0
- package/dist/domains/comet-native/native-runtime-package.d.ts +19 -0
- package/dist/domains/comet-native/native-runtime-package.d.ts.map +1 -0
- package/dist/domains/comet-native/native-runtime-package.js +76 -0
- package/dist/domains/comet-native/native-runtime-package.js.map +1 -0
- package/dist/domains/comet-native/native-schema-migration.d.ts +11 -0
- package/dist/domains/comet-native/native-schema-migration.d.ts.map +1 -0
- package/dist/domains/comet-native/native-schema-migration.js +903 -0
- package/dist/domains/comet-native/native-schema-migration.js.map +1 -0
- package/dist/domains/comet-native/native-script-entry.d.ts +12 -0
- package/dist/domains/comet-native/native-script-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-script-entry.js +44 -0
- package/dist/domains/comet-native/native-script-entry.js.map +1 -0
- package/dist/domains/comet-native/native-select-command.d.ts +3 -0
- package/dist/domains/comet-native/native-select-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-select-command.js +19 -0
- package/dist/domains/comet-native/native-select-command.js.map +1 -0
- package/dist/domains/comet-native/native-select-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-select-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-select-entry.js +4 -0
- package/dist/domains/comet-native/native-select-entry.js.map +1 -0
- package/dist/domains/comet-native/native-selection.d.ts +12 -0
- package/dist/domains/comet-native/native-selection.d.ts.map +1 -0
- package/dist/domains/comet-native/native-selection.js +61 -0
- package/dist/domains/comet-native/native-selection.js.map +1 -0
- package/dist/domains/comet-native/native-sensitive-paths.d.ts +7 -0
- package/dist/domains/comet-native/native-sensitive-paths.d.ts.map +1 -0
- package/dist/domains/comet-native/native-sensitive-paths.js +69 -0
- package/dist/domains/comet-native/native-sensitive-paths.js.map +1 -0
- package/dist/domains/comet-native/native-show-command.d.ts +3 -0
- package/dist/domains/comet-native/native-show-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-show-command.js +78 -0
- package/dist/domains/comet-native/native-show-command.js.map +1 -0
- package/dist/domains/comet-native/native-show-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-show-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-show-entry.js +4 -0
- package/dist/domains/comet-native/native-show-entry.js.map +1 -0
- package/dist/domains/comet-native/native-snapshot.d.ts +87 -0
- package/dist/domains/comet-native/native-snapshot.d.ts.map +1 -0
- package/dist/domains/comet-native/native-snapshot.js +2728 -0
- package/dist/domains/comet-native/native-snapshot.js.map +1 -0
- package/dist/domains/comet-native/native-spec-command.d.ts +3 -0
- package/dist/domains/comet-native/native-spec-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-spec-command.js +20 -0
- package/dist/domains/comet-native/native-spec-command.js.map +1 -0
- package/dist/domains/comet-native/native-spec-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-spec-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-spec-entry.js +4 -0
- package/dist/domains/comet-native/native-spec-entry.js.map +1 -0
- package/dist/domains/comet-native/native-specs.d.ts +12 -0
- package/dist/domains/comet-native/native-specs.d.ts.map +1 -0
- package/dist/domains/comet-native/native-specs.js +270 -0
- package/dist/domains/comet-native/native-specs.js.map +1 -0
- package/dist/domains/comet-native/native-status-command.d.ts +3 -0
- package/dist/domains/comet-native/native-status-command.d.ts.map +1 -0
- package/dist/domains/comet-native/native-status-command.js +26 -0
- package/dist/domains/comet-native/native-status-command.js.map +1 -0
- package/dist/domains/comet-native/native-status-discovery.d.ts +51 -0
- package/dist/domains/comet-native/native-status-discovery.d.ts.map +1 -0
- package/dist/domains/comet-native/native-status-discovery.js +317 -0
- package/dist/domains/comet-native/native-status-discovery.js.map +1 -0
- package/dist/domains/comet-native/native-status-entry.d.ts +2 -0
- package/dist/domains/comet-native/native-status-entry.d.ts.map +1 -0
- package/dist/domains/comet-native/native-status-entry.js +4 -0
- package/dist/domains/comet-native/native-status-entry.js.map +1 -0
- package/dist/domains/comet-native/native-trajectory-limits.d.ts +7 -0
- package/dist/domains/comet-native/native-trajectory-limits.d.ts.map +1 -0
- package/dist/domains/comet-native/native-trajectory-limits.js +13 -0
- package/dist/domains/comet-native/native-trajectory-limits.js.map +1 -0
- package/dist/domains/comet-native/native-trajectory-recovery.d.ts +37 -0
- package/dist/domains/comet-native/native-trajectory-recovery.d.ts.map +1 -0
- package/dist/domains/comet-native/native-trajectory-recovery.js +130 -0
- package/dist/domains/comet-native/native-trajectory-recovery.js.map +1 -0
- package/dist/domains/comet-native/native-trajectory.d.ts +16 -0
- package/dist/domains/comet-native/native-trajectory.d.ts.map +1 -0
- package/dist/domains/comet-native/native-trajectory.js +27 -0
- package/dist/domains/comet-native/native-trajectory.js.map +1 -0
- package/dist/domains/comet-native/native-transaction.d.ts +45 -0
- package/dist/domains/comet-native/native-transaction.d.ts.map +1 -0
- package/dist/domains/comet-native/native-transaction.js +857 -0
- package/dist/domains/comet-native/native-transaction.js.map +1 -0
- package/dist/domains/comet-native/native-transition-evidence.d.ts +4 -0
- package/dist/domains/comet-native/native-transition-evidence.d.ts.map +1 -0
- package/dist/domains/comet-native/native-transition-evidence.js +18 -0
- package/dist/domains/comet-native/native-transition-evidence.js.map +1 -0
- package/dist/domains/comet-native/native-transition-journal.d.ts +30 -0
- package/dist/domains/comet-native/native-transition-journal.d.ts.map +1 -0
- package/dist/domains/comet-native/native-transition-journal.js +868 -0
- package/dist/domains/comet-native/native-transition-journal.js.map +1 -0
- package/dist/domains/comet-native/native-transitions.d.ts +19 -0
- package/dist/domains/comet-native/native-transitions.d.ts.map +1 -0
- package/dist/domains/comet-native/native-transitions.js +863 -0
- package/dist/domains/comet-native/native-transitions.js.map +1 -0
- package/dist/domains/comet-native/native-types.d.ts +584 -0
- package/dist/domains/comet-native/native-types.d.ts.map +1 -0
- package/dist/domains/comet-native/native-types.js +8 -0
- package/dist/domains/comet-native/native-types.js.map +1 -0
- package/dist/domains/comet-native/native-verification-evidence.d.ts +90 -0
- package/dist/domains/comet-native/native-verification-evidence.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verification-evidence.js +536 -0
- package/dist/domains/comet-native/native-verification-evidence.js.map +1 -0
- package/dist/domains/comet-native/native-verification-receipt-runtime.d.ts +99 -0
- package/dist/domains/comet-native/native-verification-receipt-runtime.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verification-receipt-runtime.js +597 -0
- package/dist/domains/comet-native/native-verification-receipt-runtime.js.map +1 -0
- package/dist/domains/comet-native/native-verification-receipt.d.ts +79 -0
- package/dist/domains/comet-native/native-verification-receipt.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verification-receipt.js +305 -0
- package/dist/domains/comet-native/native-verification-receipt.js.map +1 -0
- package/dist/domains/comet-native/native-verification-report-v2.d.ts +12 -0
- package/dist/domains/comet-native/native-verification-report-v2.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verification-report-v2.js +141 -0
- package/dist/domains/comet-native/native-verification-report-v2.js.map +1 -0
- package/dist/domains/comet-native/native-verification-runtime.d.ts +111 -0
- package/dist/domains/comet-native/native-verification-runtime.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verification-runtime.js +520 -0
- package/dist/domains/comet-native/native-verification-runtime.js.map +1 -0
- package/dist/domains/comet-native/native-verification-scope.d.ts +141 -0
- package/dist/domains/comet-native/native-verification-scope.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verification-scope.js +1132 -0
- package/dist/domains/comet-native/native-verification-scope.js.map +1 -0
- package/dist/domains/comet-native/native-verifier-protocol.d.ts +49 -0
- package/dist/domains/comet-native/native-verifier-protocol.d.ts.map +1 -0
- package/dist/domains/comet-native/native-verifier-protocol.js +157 -0
- package/dist/domains/comet-native/native-verifier-protocol.js.map +1 -0
- package/dist/domains/comet-native/native-workspace-finish.d.ts +67 -0
- package/dist/domains/comet-native/native-workspace-finish.d.ts.map +1 -0
- package/dist/domains/comet-native/native-workspace-finish.js +383 -0
- package/dist/domains/comet-native/native-workspace-finish.js.map +1 -0
- package/dist/domains/comet-native/native-workspace-preparation.d.ts +32 -0
- package/dist/domains/comet-native/native-workspace-preparation.d.ts.map +1 -0
- package/dist/domains/comet-native/native-workspace-preparation.js +324 -0
- package/dist/domains/comet-native/native-workspace-preparation.js.map +1 -0
- package/dist/domains/comet-native/native-workspace.d.ts +90 -0
- package/dist/domains/comet-native/native-workspace.d.ts.map +1 -0
- package/dist/domains/comet-native/native-workspace.js +566 -0
- package/dist/domains/comet-native/native-workspace.js.map +1 -0
- package/dist/domains/dashboard/collector.d.ts +26 -1
- package/dist/domains/dashboard/collector.d.ts.map +1 -1
- package/dist/domains/dashboard/collector.js +609 -95
- package/dist/domains/dashboard/collector.js.map +1 -1
- package/dist/domains/dashboard/native-adapter.d.ts +201 -0
- package/dist/domains/dashboard/native-adapter.d.ts.map +1 -0
- package/dist/domains/dashboard/native-adapter.js +199 -0
- package/dist/domains/dashboard/native-adapter.js.map +1 -0
- package/dist/domains/dashboard/native-collector.d.ts +32 -0
- package/dist/domains/dashboard/native-collector.d.ts.map +1 -0
- package/dist/domains/dashboard/native-collector.js +645 -0
- package/dist/domains/dashboard/native-collector.js.map +1 -0
- package/dist/domains/dashboard/native-legacy-adapter.d.ts +37 -0
- package/dist/domains/dashboard/native-legacy-adapter.d.ts.map +1 -0
- package/dist/domains/dashboard/native-legacy-adapter.js +127 -0
- package/dist/domains/dashboard/native-legacy-adapter.js.map +1 -0
- package/dist/domains/dashboard/project-directory.d.ts +20 -0
- package/dist/domains/dashboard/project-directory.d.ts.map +1 -0
- package/dist/domains/dashboard/project-directory.js +87 -0
- package/dist/domains/dashboard/project-directory.js.map +1 -0
- package/dist/domains/dashboard/project-root.d.ts +8 -0
- package/dist/domains/dashboard/project-root.d.ts.map +1 -0
- package/dist/domains/dashboard/project-root.js +68 -0
- package/dist/domains/dashboard/project-root.js.map +1 -0
- package/dist/domains/dashboard/server.d.ts +1 -0
- package/dist/domains/dashboard/server.d.ts.map +1 -1
- package/dist/domains/dashboard/server.js +142 -11
- package/dist/domains/dashboard/server.js.map +1 -1
- package/dist/domains/dashboard/types.d.ts +55 -0
- package/dist/domains/dashboard/types.d.ts.map +1 -1
- package/dist/domains/dashboard/verify-parser.d.ts +1 -0
- package/dist/domains/dashboard/verify-parser.d.ts.map +1 -1
- package/dist/domains/dashboard/verify-parser.js +41 -31
- package/dist/domains/dashboard/verify-parser.js.map +1 -1
- package/dist/domains/dashboard/web/assets/abnfDiagram-N423BO3Z-BYya6ofk.js +1 -0
- package/dist/domains/dashboard/web/assets/arc-DJbo4TD3.js +1 -0
- package/dist/domains/dashboard/web/assets/architectureDiagram-T3A2C74G-D8uZ2IB-.js +36 -0
- package/dist/domains/dashboard/web/assets/blockDiagram-VBNYF7ZC-B16BVj-r.js +132 -0
- package/dist/domains/dashboard/web/assets/c4Diagram-5PPSVZJV-ayV208_D.js +10 -0
- package/dist/domains/dashboard/web/assets/channel-CEsrvmig.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-2GRJ4B5K-Bn5o8p3H.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-2Q5K7J3B-BnF1SI6B.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-5RXB4S5H-ETnma053.js +231 -0
- package/dist/domains/dashboard/web/assets/chunk-5VM5RSS4-Cuvy639H.js +15 -0
- package/dist/domains/dashboard/web/assets/chunk-6Q2QTUOP-DOXBzl73.js +88 -0
- package/dist/domains/dashboard/web/assets/chunk-GF5L2VYU-CfREXlEl.js +206 -0
- package/dist/domains/dashboard/web/assets/chunk-JWPE2WC7-DWIdpYTb.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-KBJHAD2P-BnqPcfZB.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-RYQCIY6F-DdJ00fZz.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-XXDRQBXY-BVDhdJyW.js +1 -0
- package/dist/domains/dashboard/web/assets/classDiagram-JCYQIIEL-CKFe0hg4.js +1 -0
- package/dist/domains/dashboard/web/assets/classDiagram-v2-OCEON4UE-CKFe0hg4.js +1 -0
- package/dist/domains/dashboard/web/assets/cose-bilkent-JH36ORCC-C-nglwHC.js +1 -0
- package/dist/domains/dashboard/web/assets/cynefin-VYW2F7L2-C8TMmrbP.js +166 -0
- package/dist/domains/dashboard/web/assets/cynefinDiagram-MW4NZA55-C3pnQnjg.js +62 -0
- package/dist/domains/dashboard/web/assets/cytoscape.esm-D3_iZ_3b.js +321 -0
- package/dist/domains/dashboard/web/assets/dagre-VZM6K2ZE-DY7UWiPI.js +4 -0
- package/dist/domains/dashboard/web/assets/defaultLocale-DX6XiGOO.js +1 -0
- package/dist/domains/dashboard/web/assets/demo-BfLsKrIL.js +45 -0
- package/dist/domains/dashboard/web/assets/diagram-7IWD3JNH-voTyDkQ5.js +30 -0
- package/dist/domains/dashboard/web/assets/diagram-B4RE2ZJO-B3VkUMck.js +3 -0
- package/dist/domains/dashboard/web/assets/diagram-LBJQPF4R-BdJzPkv8.js +24 -0
- package/dist/domains/dashboard/web/assets/diagram-Q27KOJAE-Bsq0anZx.js +24 -0
- package/dist/domains/dashboard/web/assets/diagram-UB23O5K3-CBaKRbcu.js +41 -0
- package/dist/domains/dashboard/web/assets/ebnfDiagram-BXEA7PRR-Bl9Bm5wa.js +1 -0
- package/dist/domains/dashboard/web/assets/erDiagram-JOGREHBK-DjOZzfKI.js +85 -0
- package/dist/domains/dashboard/web/assets/flowDiagram-UKHOOZJN-C21e4siK.js +156 -0
- package/dist/domains/dashboard/web/assets/ganttDiagram-PKOTCBZU-Bs62uy6-.js +292 -0
- package/dist/domains/dashboard/web/assets/gitGraphDiagram-DS77QQ5N-DQY_Ba00.js +106 -0
- package/dist/domains/dashboard/web/assets/graph-DOmOIIwC.js +1 -0
- package/dist/domains/dashboard/web/assets/index-BLv71GRj.js +10 -0
- package/dist/domains/dashboard/web/assets/index-Be4r99N4.js +257 -0
- package/dist/domains/dashboard/web/assets/index-ByVYiA6g.css +1 -0
- package/dist/domains/dashboard/web/assets/index-DlSYK56b.js +139 -0
- package/dist/domains/dashboard/web/assets/infoDiagram-6WML65LV-CNgNNG9l.js +2 -0
- package/dist/domains/dashboard/web/assets/init-Gi6I4Gst.js +1 -0
- package/dist/domains/dashboard/web/assets/ishikawaDiagram-WSZJBQD7-D9k1fTO1.js +70 -0
- package/dist/domains/dashboard/web/assets/journeyDiagram-NVQOT4AX-CHijUnPl.js +139 -0
- package/dist/domains/dashboard/web/assets/kanban-definition-27J2QSJJ-DfWlntDq.js +89 -0
- package/dist/domains/dashboard/web/assets/katex-HP8lGamR.js +257 -0
- package/dist/domains/dashboard/web/assets/layout-D-LzfAck.js +1 -0
- package/dist/domains/dashboard/web/assets/linear-o6hemX-z.js +1 -0
- package/dist/domains/dashboard/web/assets/map-DxJ2ADlA.js +1 -0
- package/dist/domains/dashboard/web/assets/marked.esm-DECB3bxB.js +64 -0
- package/dist/domains/dashboard/web/assets/mermaid.core-B1DoOsPD.js +306 -0
- package/dist/domains/dashboard/web/assets/mindmap-definition-FAOFIHXS-COMQF9Yy.js +96 -0
- package/dist/domains/dashboard/web/assets/ordinal-Cboi1Yqb.js +1 -0
- package/dist/domains/dashboard/web/assets/pegDiagram-VL7TDLO6-C_21n7Ps.js +1 -0
- package/dist/domains/dashboard/web/assets/pieDiagram-7S7Q4E2Y-D3EmoYDY.js +39 -0
- package/dist/domains/dashboard/web/assets/purify.es-Cz4mVeUR.js +3 -0
- package/dist/domains/dashboard/web/assets/quadrantDiagram-CIZ2JOQS-C3fGdiMa.js +7 -0
- package/dist/domains/dashboard/web/assets/railroadDiagram-AXF67PYL-mIsjemD-.js +1 -0
- package/dist/domains/dashboard/web/assets/requirementDiagram-LRYGKXZP-D_UMpS5e.js +84 -0
- package/dist/domains/dashboard/web/assets/sankeyDiagram-W5VNT64P-D1MwyMF6.js +40 -0
- package/dist/domains/dashboard/web/assets/sequenceDiagram-SI44F4Z6-B8jp1RxX.js +162 -0
- package/dist/domains/dashboard/web/assets/sizeCapture-X5ZJPWSS-BKioI1Ql.js +1 -0
- package/dist/domains/dashboard/web/assets/stateDiagram-OKZ733FA-B7l8_CDo.js +1 -0
- package/dist/domains/dashboard/web/assets/stateDiagram-v2-UEYNNEHI-BIxuwbeV.js +1 -0
- package/dist/domains/dashboard/web/assets/swimlanes-SLNWSIFB-951PRLYp.js +2 -0
- package/dist/domains/dashboard/web/assets/swimlanesDiagram-ULZ7WXOC-CEo6cGsy.js +8 -0
- package/dist/domains/dashboard/web/assets/timeline-definition-Z64GVDOM-CeyA2KbI.js +120 -0
- package/dist/domains/dashboard/web/assets/vennDiagram-T6HMQDX7-BHe0TZJV.js +34 -0
- package/dist/domains/dashboard/web/assets/wardleyDiagram-T6FBY63Y-CWqdSzJ-.js +78 -0
- package/dist/domains/dashboard/web/assets/xychartDiagram-ELKLHX3M-DCKiZVPV.js +7 -0
- package/dist/domains/dashboard/web/index.html +16 -16
- package/dist/domains/dashboard/workspace.d.ts +27 -0
- package/dist/domains/dashboard/workspace.d.ts.map +1 -0
- package/dist/domains/dashboard/workspace.js +108 -0
- package/dist/domains/dashboard/workspace.js.map +1 -0
- package/dist/domains/engine/protected-run-file.d.ts +15 -0
- package/dist/domains/engine/protected-run-file.d.ts.map +1 -0
- package/dist/domains/engine/protected-run-file.js +76 -0
- package/dist/domains/engine/protected-run-file.js.map +1 -0
- package/dist/domains/engine/run-store.d.ts +12 -11
- package/dist/domains/engine/run-store.d.ts.map +1 -1
- package/dist/domains/engine/run-store.js +46 -65
- package/dist/domains/engine/run-store.js.map +1 -1
- package/dist/domains/engine/state.d.ts +4 -3
- package/dist/domains/engine/state.d.ts.map +1 -1
- package/dist/domains/engine/state.js +10 -21
- package/dist/domains/engine/state.js.map +1 -1
- package/dist/domains/engine/storage-layout.d.ts +14 -0
- package/dist/domains/engine/storage-layout.d.ts.map +1 -0
- package/dist/domains/engine/storage-layout.js +32 -0
- package/dist/domains/engine/storage-layout.js.map +1 -0
- package/dist/domains/engine/storage-run.d.ts +9 -0
- package/dist/domains/engine/storage-run.d.ts.map +1 -0
- package/dist/domains/engine/storage-run.js +116 -0
- package/dist/domains/engine/storage-run.js.map +1 -0
- package/dist/domains/engine/types.d.ts +1 -1
- package/dist/domains/engine/types.d.ts.map +1 -1
- package/dist/domains/eval/index.d.ts +3 -0
- package/dist/domains/eval/index.d.ts.map +1 -1
- package/dist/domains/eval/index.js +3 -0
- package/dist/domains/eval/index.js.map +1 -1
- package/dist/domains/eval/standalone-context.d.ts +19 -0
- package/dist/domains/eval/standalone-context.d.ts.map +1 -0
- package/dist/domains/eval/standalone-context.js +107 -0
- package/dist/domains/eval/standalone-context.js.map +1 -0
- package/dist/domains/eval/standalone-static-collect.d.ts +22 -0
- package/dist/domains/eval/standalone-static-collect.d.ts.map +1 -0
- package/dist/domains/eval/standalone-static-collect.js +779 -0
- package/dist/domains/eval/standalone-static-collect.js.map +1 -0
- package/dist/domains/eval/user-environment.d.ts +6 -0
- package/dist/domains/eval/user-environment.d.ts.map +1 -0
- package/dist/domains/eval/user-environment.js +73 -0
- package/dist/domains/eval/user-environment.js.map +1 -0
- package/dist/domains/factory/package.d.ts.map +1 -1
- package/dist/domains/factory/package.js +547 -91
- package/dist/domains/factory/package.js.map +1 -1
- package/dist/domains/integrations/codegraph.d.ts +35 -2
- package/dist/domains/integrations/codegraph.d.ts.map +1 -1
- package/dist/domains/integrations/codegraph.js +466 -12
- package/dist/domains/integrations/codegraph.js.map +1 -1
- package/dist/domains/integrations/openspec.d.ts +13 -2
- package/dist/domains/integrations/openspec.d.ts.map +1 -1
- package/dist/domains/integrations/openspec.js +316 -55
- package/dist/domains/integrations/openspec.js.map +1 -1
- package/dist/domains/integrations/superpowers.d.ts +14 -1
- package/dist/domains/integrations/superpowers.d.ts.map +1 -1
- package/dist/domains/integrations/superpowers.js +93 -5
- package/dist/domains/integrations/superpowers.js.map +1 -1
- package/dist/domains/skill/find.js +8 -7
- package/dist/domains/skill/find.js.map +1 -1
- package/dist/domains/skill/hook-lifecycle.d.ts +9 -0
- package/dist/domains/skill/hook-lifecycle.d.ts.map +1 -0
- package/dist/domains/skill/hook-lifecycle.js +48 -0
- package/dist/domains/skill/hook-lifecycle.js.map +1 -0
- package/dist/domains/skill/json-object.d.ts +13 -0
- package/dist/domains/skill/json-object.d.ts.map +1 -0
- package/dist/domains/skill/json-object.js +24 -0
- package/dist/domains/skill/json-object.js.map +1 -0
- package/dist/domains/skill/languages.d.ts +2 -1
- package/dist/domains/skill/languages.d.ts.map +1 -1
- package/dist/domains/skill/languages.js +4 -1
- package/dist/domains/skill/languages.js.map +1 -1
- package/dist/domains/skill/managed-markdown.d.ts +14 -0
- package/dist/domains/skill/managed-markdown.d.ts.map +1 -0
- package/dist/domains/skill/managed-markdown.js +126 -0
- package/dist/domains/skill/managed-markdown.js.map +1 -0
- package/dist/domains/skill/platform-inspect.d.ts +15 -0
- package/dist/domains/skill/platform-inspect.d.ts.map +1 -0
- package/dist/domains/skill/platform-inspect.js +339 -0
- package/dist/domains/skill/platform-inspect.js.map +1 -0
- package/dist/domains/skill/platform-install.d.ts +65 -14
- package/dist/domains/skill/platform-install.d.ts.map +1 -1
- package/dist/domains/skill/platform-install.js +995 -233
- package/dist/domains/skill/platform-install.js.map +1 -1
- package/dist/domains/skill/project-hook-projection.d.ts +14 -0
- package/dist/domains/skill/project-hook-projection.d.ts.map +1 -0
- package/dist/domains/skill/project-hook-projection.js +41 -0
- package/dist/domains/skill/project-hook-projection.js.map +1 -0
- package/dist/domains/skill/project-instructions.d.ts +25 -0
- package/dist/domains/skill/project-instructions.d.ts.map +1 -0
- package/dist/domains/skill/project-instructions.js +85 -0
- package/dist/domains/skill/project-instructions.js.map +1 -0
- package/dist/domains/skill/uninstall.d.ts +18 -3
- package/dist/domains/skill/uninstall.d.ts.map +1 -1
- package/dist/domains/skill/uninstall.js +1081 -169
- package/dist/domains/skill/uninstall.js.map +1 -1
- package/dist/domains/workflow-contract/builtins.d.ts +6 -0
- package/dist/domains/workflow-contract/builtins.d.ts.map +1 -1
- package/dist/domains/workflow-contract/builtins.js +149 -6
- package/dist/domains/workflow-contract/builtins.js.map +1 -1
- package/dist/domains/workflow-contract/contained-atomic-write.d.ts +24 -0
- package/dist/domains/workflow-contract/contained-atomic-write.d.ts.map +1 -0
- package/dist/domains/workflow-contract/contained-atomic-write.js +256 -0
- package/dist/domains/workflow-contract/contained-atomic-write.js.map +1 -0
- package/dist/domains/workflow-contract/global-config.d.ts +10 -0
- package/dist/domains/workflow-contract/global-config.d.ts.map +1 -0
- package/dist/domains/workflow-contract/global-config.js +117 -0
- package/dist/domains/workflow-contract/global-config.js.map +1 -0
- package/dist/domains/workflow-contract/hook-target-scope.d.ts +6 -0
- package/dist/domains/workflow-contract/hook-target-scope.d.ts.map +1 -0
- package/dist/domains/workflow-contract/hook-target-scope.js +51 -0
- package/dist/domains/workflow-contract/hook-target-scope.js.map +1 -0
- package/dist/domains/workflow-contract/hook-write-policy.d.ts +9 -0
- package/dist/domains/workflow-contract/hook-write-policy.d.ts.map +1 -0
- package/dist/domains/workflow-contract/hook-write-policy.js +52 -0
- package/dist/domains/workflow-contract/hook-write-policy.js.map +1 -0
- package/dist/domains/workflow-contract/index.d.ts +15 -1
- package/dist/domains/workflow-contract/index.d.ts.map +1 -1
- package/dist/domains/workflow-contract/index.js +10 -1
- package/dist/domains/workflow-contract/index.js.map +1 -1
- package/dist/domains/workflow-contract/normalize.d.ts.map +1 -1
- package/dist/domains/workflow-contract/normalize.js +30 -11
- package/dist/domains/workflow-contract/normalize.js.map +1 -1
- package/dist/domains/workflow-contract/project-config-reader.d.ts +26 -0
- package/dist/domains/workflow-contract/project-config-reader.d.ts.map +1 -0
- package/dist/domains/workflow-contract/project-config-reader.js +54 -0
- package/dist/domains/workflow-contract/project-config-reader.js.map +1 -0
- package/dist/domains/workflow-contract/project-config-transaction.d.ts +20 -0
- package/dist/domains/workflow-contract/project-config-transaction.d.ts.map +1 -0
- package/dist/domains/workflow-contract/project-config-transaction.js +293 -0
- package/dist/domains/workflow-contract/project-config-transaction.js.map +1 -0
- package/dist/domains/workflow-contract/project-config-writer.d.ts +13 -0
- package/dist/domains/workflow-contract/project-config-writer.d.ts.map +1 -0
- package/dist/domains/workflow-contract/project-config-writer.js +143 -0
- package/dist/domains/workflow-contract/project-config-writer.js.map +1 -0
- package/dist/domains/workflow-contract/project-config.d.ts +38 -0
- package/dist/domains/workflow-contract/project-config.d.ts.map +1 -0
- package/dist/domains/workflow-contract/project-config.js +1847 -0
- package/dist/domains/workflow-contract/project-config.js.map +1 -0
- package/dist/domains/workflow-contract/project-gitignore.d.ts +9 -0
- package/dist/domains/workflow-contract/project-gitignore.d.ts.map +1 -0
- package/dist/domains/workflow-contract/project-gitignore.js +108 -0
- package/dist/domains/workflow-contract/project-gitignore.js.map +1 -0
- package/dist/domains/workflow-contract/protected-project-path.d.ts +47 -0
- package/dist/domains/workflow-contract/protected-project-path.d.ts.map +1 -0
- package/dist/domains/workflow-contract/protected-project-path.js +245 -0
- package/dist/domains/workflow-contract/protected-project-path.js.map +1 -0
- package/dist/domains/workflow-contract/types.d.ts +83 -2
- package/dist/domains/workflow-contract/types.d.ts.map +1 -1
- package/dist/domains/workflow-contract/validation.d.ts.map +1 -1
- package/dist/domains/workflow-contract/validation.js +14 -5
- package/dist/domains/workflow-contract/validation.js.map +1 -1
- package/dist/platform/fs/file-identity.d.ts +9 -0
- package/dist/platform/fs/file-identity.d.ts.map +1 -0
- package/dist/platform/fs/file-identity.js +21 -0
- package/dist/platform/fs/file-identity.js.map +1 -0
- package/dist/platform/fs/file-system.d.ts +3 -3
- package/dist/platform/fs/file-system.d.ts.map +1 -1
- package/dist/platform/fs/file-system.js +31 -17
- package/dist/platform/fs/file-system.js.map +1 -1
- package/dist/platform/fs/race-safe-read.d.ts +47 -0
- package/dist/platform/fs/race-safe-read.d.ts.map +1 -0
- package/dist/platform/fs/race-safe-read.js +132 -0
- package/dist/platform/fs/race-safe-read.js.map +1 -0
- package/dist/platform/fs/strip-bom.d.ts +10 -0
- package/dist/platform/fs/strip-bom.d.ts.map +1 -0
- package/dist/platform/fs/strip-bom.js +12 -0
- package/dist/platform/fs/strip-bom.js.map +1 -0
- package/dist/platform/fs/trusted-readonly-file.d.ts +31 -0
- package/dist/platform/fs/trusted-readonly-file.d.ts.map +1 -0
- package/dist/platform/fs/trusted-readonly-file.js +107 -0
- package/dist/platform/fs/trusted-readonly-file.js.map +1 -0
- package/dist/platform/install/detect.d.ts +6 -2
- package/dist/platform/install/detect.d.ts.map +1 -1
- package/dist/platform/install/detect.js +16 -2
- package/dist/platform/install/detect.js.map +1 -1
- package/dist/platform/install/platform-targets.d.ts +8 -0
- package/dist/platform/install/platform-targets.d.ts.map +1 -0
- package/dist/platform/install/platform-targets.js +31 -0
- package/dist/platform/install/platform-targets.js.map +1 -0
- package/dist/platform/install/platforms.d.ts +25 -1
- package/dist/platform/install/platforms.d.ts.map +1 -1
- package/dist/platform/install/platforms.js +84 -4
- package/dist/platform/install/platforms.js.map +1 -1
- package/dist/platform/install/project-registry.d.ts +38 -0
- package/dist/platform/install/project-registry.d.ts.map +1 -0
- package/dist/platform/install/project-registry.js +213 -0
- package/dist/platform/install/project-registry.js.map +1 -0
- package/dist/platform/install/skill-root-owner.d.ts +15 -0
- package/dist/platform/install/skill-root-owner.d.ts.map +1 -0
- package/dist/platform/install/skill-root-owner.js +68 -0
- package/dist/platform/install/skill-root-owner.js.map +1 -0
- package/dist/platform/paths/git-worktree.d.ts +20 -0
- package/dist/platform/paths/git-worktree.d.ts.map +1 -0
- package/dist/platform/paths/git-worktree.js +110 -0
- package/dist/platform/paths/git-worktree.js.map +1 -0
- package/dist/platform/paths/repository-layout.d.ts +16 -0
- package/dist/platform/paths/repository-layout.d.ts.map +1 -1
- package/dist/platform/paths/repository-layout.js.map +1 -1
- package/dist/platform/process/external-command.d.ts +14 -0
- package/dist/platform/process/external-command.d.ts.map +1 -0
- package/dist/platform/process/external-command.js +46 -0
- package/dist/platform/process/external-command.js.map +1 -0
- package/dist/platform/process/git.d.ts +12 -0
- package/dist/platform/process/git.d.ts.map +1 -0
- package/dist/platform/process/git.js +86 -0
- package/dist/platform/process/git.js.map +1 -0
- package/dist/platform/process/hook-read-cache.d.ts +45 -0
- package/dist/platform/process/hook-read-cache.d.ts.map +1 -0
- package/dist/platform/process/hook-read-cache.js +102 -0
- package/dist/platform/process/hook-read-cache.js.map +1 -0
- package/dist/platform/process/spawn-command.d.ts +8 -0
- package/dist/platform/process/spawn-command.d.ts.map +1 -0
- package/dist/platform/process/spawn-command.js +113 -0
- package/dist/platform/process/spawn-command.js.map +1 -0
- package/dist/platform/process/terminate-process-tree.d.ts +3 -0
- package/dist/platform/process/terminate-process-tree.d.ts.map +1 -0
- package/dist/platform/process/terminate-process-tree.js +65 -0
- package/dist/platform/process/terminate-process-tree.js.map +1 -0
- package/dist/platform/version/version.d.ts.map +1 -1
- package/dist/platform/version/version.js +9 -2
- package/dist/platform/version/version.js.map +1 -1
- package/eval/.env.example +115 -0
- package/eval/CLAUDE.md +43 -0
- package/eval/README.md +761 -0
- package/eval/langfuse/.env.example +9 -0
- package/eval/langfuse/tests/conftest.py +146 -0
- package/eval/langfuse/tests/tasks/test_tasks.py +109 -0
- package/eval/langfuse/tests/test_adapter.py +272 -0
- package/eval/langsmith/.env.example +16 -0
- package/eval/langsmith/README.md +93 -0
- package/eval/langsmith/skills/README.md +5 -0
- package/eval/langsmith/tasks/README.md +5 -0
- package/eval/langsmith/tests/conftest.py +196 -0
- package/eval/langsmith/tests/tasks/test_tasks.py +287 -0
- package/eval/langsmith/treatments/README.md +5 -0
- package/eval/local/README.md +130 -0
- package/eval/local/regression_baseline.json +13 -0
- package/eval/local/report-style-demo-assets/quality_cost.en.svg +41 -0
- package/eval/local/report-style-demo-assets/quality_cost.zh.svg +41 -0
- package/eval/local/report-style-demo-assets/rubric_delta.en.svg +56 -0
- package/eval/local/report-style-demo-assets/rubric_delta.zh.svg +56 -0
- package/eval/local/report-style-demo-assets/task_outcomes.en.svg +46 -0
- package/eval/local/report-style-demo-assets/task_outcomes.zh.svg +46 -0
- package/eval/local/report-style-demo.html +867 -0
- package/eval/local/scripts/compare_baselines.py +1144 -0
- package/eval/local/scripts/generate_report_style_demo_charts.py +322 -0
- package/eval/local/scripts/regression_check.py +170 -0
- package/eval/local/scripts/render_readme_benchmark_figures.py +416 -0
- package/eval/local/scripts/rescore_rubric.py +120 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/SKILL.md +282 -0
- package/{assets/skills/comet → eval/local/skills/benchmarks/039-release/comet-classic-039}/reference/auto-transition.md +1 -1
- package/{assets/skills/comet → eval/local/skills/benchmarks/039-release/comet-classic-039}/reference/comet-yaml-fields.md +4 -11
- package/{assets/skills/comet → eval/local/skills/benchmarks/039-release/comet-classic-039}/reference/context-recovery.md +2 -12
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/debug-gate.md +17 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/decision-point.md +20 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/subagent-dispatch.md +113 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/rules/comet-phase-guard.md +125 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-archive.sh +311 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-env.sh +110 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-guard.sh +778 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-handoff.sh +390 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-hook-guard.sh +336 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-state.sh +1338 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-yaml-validate.sh +225 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-archive/SKILL.md +100 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-build/SKILL.md +304 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-design/SKILL.md +263 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-hotfix/SKILL.md +204 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-open/SKILL.md +220 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-tweak/SKILL.md +180 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-verify/SKILL.md +234 -0
- package/eval/local/skills/benchmarks/040-beta/comet/SKILL.md +261 -0
- package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/comet-yaml-fields.md +0 -2
- package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/subagent-dispatch.md +2 -2
- package/eval/local/skills/benchmarks/040-beta/comet/rules/comet-phase-guard.md +130 -0
- package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/checks.yaml +6 -0
- package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/guardrails.yaml +13 -0
- package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/skill.yaml +122 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-archive.mjs +9515 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-env.mjs +15 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-guard.mjs +10173 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-handoff.mjs +9491 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-hook-guard.mjs +9175 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-intent.mjs +404 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-state.mjs +9273 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-yaml-validate.mjs +7627 -0
- package/eval/local/skills/benchmarks/040-beta/comet-archive/SKILL.md +110 -0
- package/eval/local/skills/benchmarks/040-beta/comet-build/SKILL.md +315 -0
- package/eval/local/skills/benchmarks/040-beta/comet-design/SKILL.md +260 -0
- package/eval/local/skills/benchmarks/040-beta/comet-hotfix/SKILL.md +190 -0
- package/eval/local/skills/benchmarks/040-beta/comet-open/SKILL.md +210 -0
- package/eval/local/skills/benchmarks/040-beta/comet-tweak/SKILL.md +183 -0
- package/eval/local/skills/benchmarks/040-beta/comet-verify/SKILL.md +230 -0
- package/eval/local/skills/benchmarks/dependency/claude-md/comet-workflow/CLAUDE.md +12 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-apply-change/SKILL.md +156 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-archive-change/SKILL.md +114 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-bulk-archive-change/SKILL.md +246 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-continue-change/SKILL.md +118 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-explore/SKILL.md +288 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-ff-change/SKILL.md +101 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-new-change/SKILL.md +74 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-onboard/SKILL.md +554 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-propose/SKILL.md +110 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-sync-specs/SKILL.md +138 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-verify-change/SKILL.md +168 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/SKILL.md +159 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/frame-template.html +213 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/helper.js +167 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/server.cjs +723 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/start-server.sh +209 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/stop-server.sh +120 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/spec-document-reviewer-prompt.md +49 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/visual-companion.md +291 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/dispatching-parallel-agents/SKILL.md +185 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/executing-plans/SKILL.md +70 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/finishing-a-development-branch/SKILL.md +241 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/receiving-code-review/SKILL.md +213 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/requesting-code-review/SKILL.md +103 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/requesting-code-review/code-reviewer.md +172 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/SKILL.md +418 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/implementer-prompt.md +139 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/review-package +44 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/sdd-workspace +22 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/task-brief +40 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/task-reviewer-prompt.md +188 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/CREATION-LOG.md +119 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/SKILL.md +296 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/condition-based-waiting-example.ts +158 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/condition-based-waiting.md +115 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/defense-in-depth.md +122 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/find-polluter.sh +63 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/root-cause-tracing.md +169 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-academic.md +14 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-1.md +58 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-2.md +68 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-3.md +69 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/test-driven-development/SKILL.md +371 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/test-driven-development/testing-anti-patterns.md +299 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/using-git-worktrees/SKILL.md +202 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/verification-before-completion/SKILL.md +139 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-plans/SKILL.md +174 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-plans/plan-document-reviewer-prompt.md +49 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/SKILL.md +689 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/anthropic-best-practices.md +1150 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/examples/CLAUDE_MD_TESTING.md +189 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/graphviz-conventions.dot +172 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/persuasion-principles.md +187 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/render-graphs.js +168 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/testing-skills-with-subagents.md +384 -0
- package/eval/local/tasks/authoring-skill-smoke/environment/Dockerfile +14 -0
- package/eval/local/tasks/authoring-skill-smoke/instruction.md +4 -0
- package/eval/local/tasks/authoring-skill-smoke/task.toml +21 -0
- package/eval/local/tasks/authoring-skill-smoke/validation/test_authoring_skill_smoke.py +54 -0
- package/eval/local/tasks/comet-agent-memory-routing/environment/Dockerfile +26 -0
- package/eval/local/tasks/comet-agent-memory-routing/environment/agent_system.py +40 -0
- package/eval/local/tasks/comet-agent-memory-routing/environment/test_agent_system.py +29 -0
- package/eval/local/tasks/comet-agent-memory-routing/instruction.md +20 -0
- package/eval/local/tasks/comet-agent-memory-routing/task.toml +24 -0
- package/eval/local/tasks/comet-agent-memory-routing/validation/test_agent_memory_routing.py +24 -0
- package/eval/local/tasks/comet-api-cache-ttl/environment/Dockerfile +35 -0
- package/eval/local/tasks/comet-api-cache-ttl/environment/cache.py +39 -0
- package/eval/local/tasks/comet-api-cache-ttl/environment/test_cache.py +81 -0
- package/eval/local/tasks/comet-api-cache-ttl/instruction.md +33 -0
- package/eval/local/tasks/comet-api-cache-ttl/task.toml +24 -0
- package/eval/local/tasks/comet-api-cache-ttl/validation/test_api_cache_ttl.py +58 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/environment/.include-current-comet-cli +1 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/environment/Dockerfile +45 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/environment/current-comet-package.json +22 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/environment/current-comet.sh +14 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/environment/test_wordcount.py +29 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/environment/wordcount.py +40 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/instruction.md +18 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/task.toml +23 -0
- package/eval/local/tasks/comet-classic-layout-lifecycle/validation/test_classic_layout_lifecycle.py +604 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/cli.py +13 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/counting.py +13 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/test_textkit.py +21 -0
- package/eval/local/tasks/comet-cross-file-refactor/instruction.md +16 -0
- package/eval/local/tasks/comet-cross-file-refactor/task.toml +20 -0
- package/eval/local/tasks/comet-cross-file-refactor/validation/test_cross_file_refactor.py +20 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/app.py +8 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/settings_loader.py +14 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/test_settings_service.py +19 -0
- package/eval/local/tasks/comet-dependency-confusion/instruction.md +18 -0
- package/eval/local/tasks/comet-dependency-confusion/task.toml +24 -0
- package/eval/local/tasks/comet-dependency-confusion/validation/test_dependency_confusion.py +24 -0
- package/eval/local/tasks/comet-fix-median/environment/Dockerfile +46 -0
- package/eval/local/tasks/comet-fix-median/environment/stats.py +54 -0
- package/eval/local/tasks/comet-fix-median/environment/test_stats.py +68 -0
- package/eval/local/tasks/comet-fix-median/instruction.md +25 -0
- package/eval/local/tasks/comet-fix-median/task.toml +24 -0
- package/eval/local/tasks/comet-fix-median/validation/test_fix_median.py +60 -0
- package/eval/local/tasks/comet-framework-selection/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-framework-selection/environment/architecture.py +12 -0
- package/eval/local/tasks/comet-framework-selection/environment/test_architecture.py +30 -0
- package/eval/local/tasks/comet-framework-selection/instruction.md +22 -0
- package/eval/local/tasks/comet-framework-selection/task.toml +24 -0
- package/eval/local/tasks/comet-framework-selection/validation/test_framework_selection.py +24 -0
- package/eval/local/tasks/comet-full-workflow/environment/Dockerfile +49 -0
- package/eval/local/tasks/comet-full-workflow/environment/test_wordcount.py +29 -0
- package/eval/local/tasks/comet-full-workflow/environment/wordcount.py +40 -0
- package/eval/local/tasks/comet-full-workflow/instruction.md +18 -0
- package/eval/local/tasks/comet-full-workflow/task.toml +24 -0
- package/eval/local/tasks/comet-full-workflow/validation/test_full_workflow.py +171 -0
- package/eval/local/tasks/comet-graph-execution-review/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-graph-execution-review/environment/pipeline.py +19 -0
- package/eval/local/tasks/comet-graph-execution-review/environment/test_pipeline.py +53 -0
- package/eval/local/tasks/comet-graph-execution-review/instruction.md +21 -0
- package/eval/local/tasks/comet-graph-execution-review/task.toml +24 -0
- package/eval/local/tasks/comet-graph-execution-review/validation/test_graph_execution_review.py +24 -0
- package/eval/local/tasks/comet-human-approval-flow/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-human-approval-flow/environment/approvals.py +17 -0
- package/eval/local/tasks/comet-human-approval-flow/environment/test_approvals.py +30 -0
- package/eval/local/tasks/comet-human-approval-flow/instruction.md +15 -0
- package/eval/local/tasks/comet-human-approval-flow/task.toml +20 -0
- package/eval/local/tasks/comet-human-approval-flow/validation/test_human_approval_flow.py +20 -0
- package/eval/local/tasks/comet-layered-streaming-fix/environment/Dockerfile +17 -0
- package/eval/local/tasks/comet-layered-streaming-fix/environment/chat_app.py +19 -0
- package/eval/local/tasks/comet-layered-streaming-fix/environment/test_chat_app.py +15 -0
- package/eval/local/tasks/comet-layered-streaming-fix/instruction.md +14 -0
- package/eval/local/tasks/comet-layered-streaming-fix/task.toml +20 -0
- package/eval/local/tasks/comet-layered-streaming-fix/validation/test_layered_streaming_fix.py +20 -0
- package/eval/local/tasks/comet-native-clarification/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-clarification/environment/test_wordcount.py +25 -0
- package/eval/local/tasks/comet-native-clarification/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-clarification/instruction.md +7 -0
- package/eval/local/tasks/comet-native-clarification/task.toml +40 -0
- package/eval/local/tasks/comet-native-clarification/validation/test_native_clarification.py +181 -0
- package/eval/local/tasks/comet-native-clarification-depth/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-clarification-depth/environment/test_wordcount.py +25 -0
- package/eval/local/tasks/comet-native-clarification-depth/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-clarification-depth/instruction.md +17 -0
- package/eval/local/tasks/comet-native-clarification-depth/task.toml +47 -0
- package/eval/local/tasks/comet-native-clarification-depth/validation/test_native_clarification_depth.py +615 -0
- package/eval/local/tasks/comet-native-clarification-modes/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-clarification-modes/environment/test_wordcount.py +25 -0
- package/eval/local/tasks/comet-native-clarification-modes/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-clarification-modes/instruction.md +13 -0
- package/eval/local/tasks/comet-native-clarification-modes/task.toml +39 -0
- package/eval/local/tasks/comet-native-clarification-modes/validation/test_native_clarification_modes.py +229 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/Dockerfile +26 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/docs/comet/changes/add-character-counting/brief.md +17 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/docs/comet/changes/add-character-counting/comet-state.yaml +12 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/docs/comet/changes/add-character-counting/runtime/transition.json +69 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/docs/comet/changes/add-character-counting/specs/character-counting/spec.md +3 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/test_wordcount.py +20 -0
- package/eval/local/tasks/comet-native-interrupted-transition/environment/wordcount.py +27 -0
- package/eval/local/tasks/comet-native-interrupted-transition/instruction.md +7 -0
- package/eval/local/tasks/comet-native-interrupted-transition/task.toml +35 -0
- package/eval/local/tasks/comet-native-interrupted-transition/validation/test_native_interrupted_transition.py +62 -0
- package/eval/local/tasks/comet-native-repository-fact/environment/Dockerfile +24 -0
- package/eval/local/tasks/comet-native-repository-fact/environment/PRODUCT_RULES.md +3 -0
- package/eval/local/tasks/comet-native-repository-fact/environment/test_wordcount.py +20 -0
- package/eval/local/tasks/comet-native-repository-fact/environment/wordcount.py +27 -0
- package/eval/local/tasks/comet-native-repository-fact/instruction.md +7 -0
- package/eval/local/tasks/comet-native-repository-fact/task.toml +35 -0
- package/eval/local/tasks/comet-native-repository-fact/validation/test_native_repository_fact.py +59 -0
- package/eval/local/tasks/comet-native-wave-b-decision-resume/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-wave-b-decision-resume/environment/test_wordcount.py +27 -0
- package/eval/local/tasks/comet-native-wave-b-decision-resume/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-wave-b-decision-resume/instruction.md +13 -0
- package/eval/local/tasks/comet-native-wave-b-decision-resume/task.toml +40 -0
- package/eval/local/tasks/comet-native-wave-b-decision-resume/validation/test_native_wave_b_decision_resume.py +199 -0
- package/eval/local/tasks/comet-native-wave-c-verification-integrity/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-wave-c-verification-integrity/environment/test_wordcount.py +27 -0
- package/eval/local/tasks/comet-native-wave-c-verification-integrity/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-wave-c-verification-integrity/instruction.md +11 -0
- package/eval/local/tasks/comet-native-wave-c-verification-integrity/task.toml +36 -0
- package/eval/local/tasks/comet-native-wave-c-verification-integrity/validation/test_native_wave_c_verification_integrity.py +124 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/environment/test_wordcount.py +27 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/instruction.md +11 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/task.toml +37 -0
- package/eval/local/tasks/comet-native-wave-d-stagnation-stop/validation/test_native_wave_d_stagnation_stop.py +215 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/environment/.include-trusted-native-runtime +1 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/environment/Dockerfile +25 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/environment/test_wordcount.py +27 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/instruction.md +9 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/task.toml +37 -0
- package/eval/local/tasks/comet-native-wave-e-parallel-safety/validation/test_native_wave_e_parallel_safety.py +205 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/environment/.include-current-comet-cli +1 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/environment/Dockerfile +29 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/environment/current-comet-package.json +20 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/environment/current-comet.sh +14 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/environment/test_wordcount.py +27 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/instruction.md +7 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/task.toml +36 -0
- package/eval/local/tasks/comet-native-wave-f-dashboard-readonly/validation/test_native_wave_f_dashboard_readonly.py +125 -0
- package/eval/local/tasks/comet-native-workflow/environment/.include-trusted-native-runtime +1 -0
- package/eval/local/tasks/comet-native-workflow/environment/Dockerfile +26 -0
- package/eval/local/tasks/comet-native-workflow/environment/test_wordcount.py +25 -0
- package/eval/local/tasks/comet-native-workflow/environment/wordcount.py +36 -0
- package/eval/local/tasks/comet-native-workflow/instruction.md +24 -0
- package/eval/local/tasks/comet-native-workflow/task.toml +38 -0
- package/eval/local/tasks/comet-native-workflow/validation/test_native_workflow.py +250 -0
- package/eval/local/tasks/comet-noise-distractor/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-noise-distractor/environment/distractor.md +4 -0
- package/eval/local/tasks/comet-noise-distractor/environment/invoice.py +5 -0
- package/eval/local/tasks/comet-noise-distractor/environment/test_invoice.py +18 -0
- package/eval/local/tasks/comet-noise-distractor/instruction.md +17 -0
- package/eval/local/tasks/comet-noise-distractor/task.toml +20 -0
- package/eval/local/tasks/comet-noise-distractor/validation/test_noise_distractor.py +20 -0
- package/eval/local/tasks/comet-observability-env-template/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-observability-env-template/environment/env_writer.py +17 -0
- package/eval/local/tasks/comet-observability-env-template/environment/test_env_writer.py +17 -0
- package/eval/local/tasks/comet-observability-env-template/instruction.md +20 -0
- package/eval/local/tasks/comet-observability-env-template/task.toml +20 -0
- package/eval/local/tasks/comet-observability-env-template/validation/test_observability_env_template.py +46 -0
- package/eval/local/tasks/comet-perf-dedupe/environment/Dockerfile +35 -0
- package/eval/local/tasks/comet-perf-dedupe/environment/dedupe.py +22 -0
- package/eval/local/tasks/comet-perf-dedupe/environment/test_dedupe.py +43 -0
- package/eval/local/tasks/comet-perf-dedupe/instruction.md +25 -0
- package/eval/local/tasks/comet-perf-dedupe/task.toml +24 -0
- package/eval/local/tasks/comet-perf-dedupe/validation/test_perf_dedupe.py +40 -0
- package/eval/local/tasks/comet-persistence-threading/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-persistence-threading/environment/session_store.py +15 -0
- package/eval/local/tasks/comet-persistence-threading/environment/test_session_store.py +21 -0
- package/eval/local/tasks/comet-persistence-threading/instruction.md +14 -0
- package/eval/local/tasks/comet-persistence-threading/task.toml +20 -0
- package/eval/local/tasks/comet-persistence-threading/validation/test_persistence_threading.py +20 -0
- package/eval/local/tasks/comet-refactor-counter/environment/Dockerfile +35 -0
- package/eval/local/tasks/comet-refactor-counter/environment/test_text_processor.py +57 -0
- package/eval/local/tasks/comet-refactor-counter/environment/text_processor.py +49 -0
- package/eval/local/tasks/comet-refactor-counter/instruction.md +31 -0
- package/eval/local/tasks/comet-refactor-counter/task.toml +24 -0
- package/eval/local/tasks/comet-refactor-counter/validation/test_refactor_counter.py +75 -0
- package/eval/local/tasks/comet-robust-config/environment/Dockerfile +35 -0
- package/eval/local/tasks/comet-robust-config/environment/config_loader.py +56 -0
- package/eval/local/tasks/comet-robust-config/environment/test_config_loader.py +72 -0
- package/eval/local/tasks/comet-robust-config/instruction.md +30 -0
- package/eval/local/tasks/comet-robust-config/task.toml +24 -0
- package/eval/local/tasks/comet-robust-config/validation/test_robust_config.py +37 -0
- package/eval/local/tasks/generic-skill-smoke/environment/Dockerfile +14 -0
- package/eval/local/tasks/generic-skill-smoke/instruction.md +7 -0
- package/eval/local/tasks/generic-skill-smoke/task.toml +21 -0
- package/eval/local/tasks/generic-skill-smoke/validation/test_generic_skill_smoke.py +41 -0
- package/eval/local/tasks/index.yaml +229 -0
- package/eval/local/tasks/workflow-overlay-contract/environment/Dockerfile +16 -0
- package/eval/local/tasks/workflow-overlay-contract/instruction.md +5 -0
- package/eval/local/tasks/workflow-overlay-contract/task.toml +21 -0
- package/eval/local/tasks/workflow-overlay-contract/validation/test_workflow_overlay_contract.py +235 -0
- package/eval/local/tasks/workflow-route-conformance/environment/Dockerfile +14 -0
- package/eval/local/tasks/workflow-route-conformance/instruction.md +5 -0
- package/eval/local/tasks/workflow-route-conformance/task.toml +21 -0
- package/eval/local/tasks/workflow-route-conformance/validation/test_workflow_route_conformance.py +84 -0
- package/eval/local/tests/conftest.py +2364 -0
- package/eval/local/tests/scaffold/fixtures/custom-agent/adapter.yaml +19 -0
- package/eval/local/tests/scaffold/test_agents.py +76 -0
- package/eval/local/tests/scaffold/test_aligned_comparison.py +1082 -0
- package/eval/local/tests/scaffold/test_attribution.py +48 -0
- package/eval/local/tests/scaffold/test_auto_tasks.py +350 -0
- package/eval/local/tests/scaffold/test_compare_baselines.py +999 -0
- package/eval/local/tests/scaffold/test_conftest_helpers.py +1286 -0
- package/eval/local/tests/scaffold/test_custom_agents.py +174 -0
- package/eval/local/tests/scaffold/test_eval_context.py +217 -0
- package/eval/local/tests/scaffold/test_evidence.py +15 -0
- package/eval/local/tests/scaffold/test_execution.py +364 -0
- package/eval/local/tests/scaffold/test_inline_tasks.py +56 -0
- package/eval/local/tests/scaffold/test_langfuse_trajectory.py +263 -0
- package/eval/local/tests/scaffold/test_langsmith_conftest.py +307 -0
- package/eval/local/tests/scaffold/test_logging.py +539 -0
- package/eval/local/tests/scaffold/test_manifest_tasks.py +76 -0
- package/eval/local/tests/scaffold/test_manifests.py +530 -0
- package/eval/local/tests/scaffold/test_native_aligned_eval.py +481 -0
- package/eval/local/tests/scaffold/test_native_wave_evaluations.py +116 -0
- package/eval/local/tests/scaffold/test_profiles.py +1207 -0
- package/eval/local/tests/scaffold/test_regression_check.py +19 -0
- package/eval/local/tests/scaffold/test_report_style_demo_charts.py +45 -0
- package/eval/local/tests/scaffold/test_sample_quality.py +268 -0
- package/eval/local/tests/scaffold/test_task_resolution.py +212 -0
- package/eval/local/tests/scaffold/test_tasks.py +996 -0
- package/eval/local/tests/scaffold/test_treatments.py +353 -0
- package/eval/local/tests/scaffold/test_utils.py +1108 -0
- package/eval/local/tests/tasks/__init__.py +1 -0
- package/eval/local/tests/tasks/test_tasks.py +676 -0
- package/eval/local/tests/tasks/test_validation_scripts.py +787 -0
- package/eval/local/treatments/comet/comet_classic_docs_layout.yaml +121 -0
- package/eval/local/treatments/comet/comet_full_039.yaml +141 -0
- package/eval/local/treatments/comet/comet_full_040_beta.yaml +140 -0
- package/eval/local/treatments/comet/comet_native_phase1.yaml +69 -0
- package/eval/local/treatments/common/control.yaml +4 -0
- package/eval/pyproject.toml +46 -0
- package/eval/report-html-config.json +6 -0
- package/eval/scaffold/__init__.py +73 -0
- package/eval/scaffold/docker/agent-overlay.Dockerfile +41 -0
- package/eval/scaffold/python/__init__.py +203 -0
- package/eval/scaffold/python/agents.py +239 -0
- package/eval/scaffold/python/aligned_comparison.py +1761 -0
- package/eval/scaffold/python/attribution.py +43 -0
- package/eval/scaffold/python/auto_tasks.py +700 -0
- package/eval/scaffold/python/custom_agents.py +172 -0
- package/eval/scaffold/python/eval_context.py +236 -0
- package/eval/scaffold/python/eval_schema.py +135 -0
- package/eval/scaffold/python/evidence.py +38 -0
- package/eval/scaffold/python/execution.py +544 -0
- package/eval/scaffold/python/external_data_handler.py +18 -0
- package/eval/scaffold/python/generated_task_cache.py +230 -0
- package/eval/scaffold/python/generic_llm_judge.py +264 -0
- package/eval/scaffold/python/inline_tasks.py +118 -0
- package/eval/scaffold/python/judge_config.py +289 -0
- package/eval/scaffold/python/langfuse_adapter.py +1023 -0
- package/eval/scaffold/python/llm_judge.py +245 -0
- package/eval/scaffold/python/logging.py +944 -0
- package/eval/scaffold/python/manifest_tasks.py +69 -0
- package/eval/scaffold/python/manifests.py +552 -0
- package/eval/scaffold/python/native_eval.py +148 -0
- package/eval/scaffold/python/paper_charts.py +25 -0
- package/eval/scaffold/python/pass_at_k.py +106 -0
- package/eval/scaffold/python/paths.py +66 -0
- package/eval/scaffold/python/profiles.py +127 -0
- package/eval/scaffold/python/report_outputs.py +1573 -0
- package/eval/scaffold/python/sample_quality.py +382 -0
- package/eval/scaffold/python/schema.py +39 -0
- package/eval/scaffold/python/skill_parser.py +469 -0
- package/eval/scaffold/python/task_resolution.py +176 -0
- package/eval/scaffold/python/tasks.py +356 -0
- package/eval/scaffold/python/treatments.py +271 -0
- package/eval/scaffold/python/utils.py +616 -0
- package/eval/scaffold/python/validation/__init__.py +31 -0
- package/eval/scaffold/python/validation/authoring_rubric.py +328 -0
- package/eval/scaffold/python/validation/comet_workflow.py +227 -0
- package/eval/scaffold/python/validation/core.py +310 -0
- package/eval/scaffold/python/validation/dataset.py +337 -0
- package/eval/scaffold/python/validation/docker.py +106 -0
- package/eval/scaffold/python/validation/evaluator.py +549 -0
- package/eval/scaffold/python/validation/generic_rubric.py +207 -0
- package/eval/scaffold/python/validation/native_v4.py +249 -0
- package/eval/scaffold/python/validation/native_wave.py +2153 -0
- package/eval/scaffold/python/validation/native_workflow.py +353 -0
- package/eval/scaffold/python/validation/rubric.py +908 -0
- package/eval/scaffold/python/validation/runner.py +237 -0
- package/eval/scaffold/python/validation/scripts.py +58 -0
- package/eval/scaffold/python/validation/tracing.py +313 -0
- package/eval/scaffold/shell/agent-runtime-config.sh +88 -0
- package/eval/scaffold/shell/common.sh +126 -0
- package/eval/scaffold/shell/completion-point.sh +18 -0
- package/eval/scaffold/shell/decision-point.sh +35 -0
- package/eval/scaffold/shell/docker.sh +1129 -0
- package/eval/scaffold/shell/run-agent-runtime.sh +49 -0
- package/eval/scaffold/shell/run-claude-loop.sh +373 -0
- package/eval/scaffold/shell/setup.sh +259 -0
- package/eval/schemas/comet.eval/v1alpha1.schema.json +244 -0
- package/eval/simulator-instruction.md +9 -0
- package/package.json +59 -8
- package/assets/skills/comet/reference/decision-point.md +0 -31
- package/assets/skills/comet/reference/scripts.md +0 -69
- package/dist/domains/dashboard/web/assets/demo-DQyfozbi.js +0 -1
- package/dist/domains/dashboard/web/assets/index-BcO3_QlT.css +0 -1
- package/dist/domains/dashboard/web/assets/index-CjU3VXVu.js +0 -22
- /package/{assets/skills/comet → eval/local/skills/benchmarks/039-release/comet-classic-039}/reference/dirty-worktree.md +0 -0
- /package/{assets/skills/comet → eval/local/skills/benchmarks/039-release/comet-classic-039}/reference/file-structure.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/auto-transition.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/context-recovery.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/debug-gate.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/decision-point.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/dirty-worktree.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/file-structure.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/intent-frame.md +0 -0
- /package/{assets/skills-zh → eval/local/skills/benchmarks/040-beta}/comet/reference/scripts.md +0 -0
package/eval/README.md
ADDED
|
@@ -0,0 +1,761 @@
|
|
|
1
|
+
# Comet Skill 评估
|
|
2
|
+
|
|
3
|
+
基于 pytest 的 Skill 评估框架,用于量化衡量 Comet 工作流和任意本地 Skill 的执行质量。
|
|
4
|
+
|
|
5
|
+
## 目录结构
|
|
6
|
+
|
|
7
|
+
```text
|
|
8
|
+
eval/
|
|
9
|
+
├── scaffold/ # 共享 Python 运行器、Docker 辅助、加载器、验证器
|
|
10
|
+
│ └── python/
|
|
11
|
+
│ ├── validation/
|
|
12
|
+
│ │ ├── generic_rubric.py # 通用 rubric(7 维度 + LLM judge)
|
|
13
|
+
│ │ ├── rubric.py # Comet 工作流 rubric(9 维度 + LLM judge)
|
|
14
|
+
│ │ ├── authoring_rubric.py # /comet-any 生成包 rubric(11 维度)
|
|
15
|
+
│ │ └── core.py # 验证器基础工具
|
|
16
|
+
│ ├── generic_llm_judge.py # 通用 LLM-as-judge 模块
|
|
17
|
+
│ ├── llm_judge.py # Comet LLM-as-judge 模块
|
|
18
|
+
│ ├── profiles.py # 评估 profile 注册表
|
|
19
|
+
│ ├── tasks.py # 任务加载器(task.toml 解析)
|
|
20
|
+
│ └── treatments.py # Treatment 加载器
|
|
21
|
+
├── local/ # 本地评估套件
|
|
22
|
+
│ ├── tasks/ # 任务定义(每个子目录是一个任务)
|
|
23
|
+
│ ├── treatments/ # Treatment 配置(注入哪些 Skill)
|
|
24
|
+
│ ├── tests/ # pytest 测试入口
|
|
25
|
+
│ └── logs/ # 评估结果日志
|
|
26
|
+
├── langsmith/ # LangSmith 评估入口(可选)
|
|
27
|
+
└── langfuse/ # Langfuse 评估入口(可选)
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## 环境配置
|
|
31
|
+
|
|
32
|
+
普通 npm 用户不需要进入 Comet 源码目录。首次运行 `comet eval` 时,CLI 会在用户目录自动创建完整配置模板:
|
|
33
|
+
|
|
34
|
+
- Windows:`%USERPROFILE%\.comet\eval\.env`
|
|
35
|
+
- macOS/Linux:`~/.comet/eval/.env`
|
|
36
|
+
|
|
37
|
+
模板包含主任务(Bench)、独立 Judge、Claude Code、Codex、Qoder、CodeBuddy、LangSmith 和 Langfuse 的全部可配置参数。CLI 只会创建缺失文件,不会覆盖已有 `.env`;当前进程环境变量优先于文件内容。源码仓库下的 `eval/.env` 仅用于 Comet harness 开发和测试,不是普通用户配置入口。
|
|
38
|
+
|
|
39
|
+
### 前置依赖
|
|
40
|
+
|
|
41
|
+
- Python 3.11+
|
|
42
|
+
- Docker
|
|
43
|
+
- 一个受支持的评估 Agent CLI:Claude Code、Codex、Qoder 或 CodeBuddy(运行时会在 Docker 镜像中准备对应 CLI)
|
|
44
|
+
- `uv`(推荐)或 pip
|
|
45
|
+
|
|
46
|
+
### 安装
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
cd eval
|
|
50
|
+
uv sync
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
### 环境变量
|
|
54
|
+
|
|
55
|
+
已发布的 `comet eval` 不需要用户拉取源码。评估运行会先读取当前 shell 的环境变量,再按需加载用户目录下、与自定义 Agent 适配器并列的 `~/.comet/eval/.env`(Windows:`%USERPROFILE%\.comet\eval\.env`)。录屏或演示时可以不创建 `.env`,把密钥放在系统环境变量里。
|
|
56
|
+
|
|
57
|
+
如需本地文件配置,可参考 `.env.example`,创建用户级文件:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
mkdir -p ~/.comet/eval
|
|
61
|
+
cp .env.example ~/.comet/eval/.env
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
进程环境变量优先于用户 `.env`;显式 CLI 或 manifest 配置优先于环境默认值。空值不会覆盖已有的有效进程变量。
|
|
65
|
+
|
|
66
|
+
| 变量 | 必填 | 说明 |
|
|
67
|
+
| ----------------------------------------------------------------- | ---- | --------------------------------------------------------------------------------------------------- |
|
|
68
|
+
| `BENCH_EVAL_AGENT` | ❌ | 未通过 CLI 或 manifest 指定时的默认 Agent |
|
|
69
|
+
| `BENCH_API_KEY` / `BENCH_BASE_URL` / `BENCH_MODEL` | ✅\* | 主任务统一凭据、API 地址和模型;会映射到所选 Agent 的原生配置 |
|
|
70
|
+
| `BENCH_JUDGE_AGENT` / `BENCH_JUDGE_API_KEY` / `BENCH_JUDGE_BASE_URL` / `BENCH_JUDGE_MODEL` | Judge | LLM-as-judge 的独立 Agent、凭据、API 地址和模型;不会继承主任务配置 |
|
|
71
|
+
| `ANTHROPIC_API_KEY` | ✅\* | Claude Code 原生 API 密钥;显式设置时优先于 `BENCH_API_KEY` |
|
|
72
|
+
| `BENCH_CC_MODEL` / `BENCH_CC_VERSION` | ❌ | Claude 模型和 CLI 版本覆盖(默认用 CLI 配置) |
|
|
73
|
+
| `OPENAI_API_KEY` / `CODEX_API_KEY` | Codex | 使用 `--agent codex` 时的 Codex 认证 |
|
|
74
|
+
| `OPENAI_BASE_URL` / `OPENAI_MODEL` / `CODEX_BASE_URL` / `CODEX_MODEL` / `BENCH_CODEX_VERSION` | ❌ | Codex 原生地址、模型和 CLI 版本覆盖 |
|
|
75
|
+
| `QODER_PERSONAL_ACCESS_TOKEN` | Qoder | 使用 `--agent qoder` 时的 Qoder 认证 |
|
|
76
|
+
| `QODER_BASE_URL` / `QODER_MODEL` / `BENCH_QODER_VERSION` | ❌ | Qoder 原生地址、模型和 CLI 版本覆盖 |
|
|
77
|
+
| `CODEBUDDY_API_KEY` / `CODEBUDDY_AUTH_TOKEN` | CodeBuddy | 使用 `--agent codebuddy` 时的 CodeBuddy 认证 |
|
|
78
|
+
| `CODEBUDDY_BASE_URL` / `CODEBUDDY_MODEL` / `BENCH_CODEBUDDY_VERSION` | ❌ | CodeBuddy API endpoint、模型和 CLI 版本覆盖 |
|
|
79
|
+
| `CODEBUDDY_SMALL_FAST_MODEL` / `CODEBUDDY_BIG_SLOW_MODEL` / `CODEBUDDY_CODE_SUBAGENT_MODEL` | ❌ | CodeBuddy 的 fast、reasoning 和 code-subagent 模型覆盖 |
|
|
80
|
+
| `CODEBUDDY_CUSTOM_HEADERS` / `CODEBUDDY_INTERNET_ENVIRONMENT` | ❌ | CodeBuddy 的自定义请求头和联网环境配置 |
|
|
81
|
+
| `BENCH_SIMULATOR_PROMPT_FILE` | ❌ | 显式覆盖任务级用户模拟器提示词;任务未配置时回退到 `eval/simulator-instruction.md` |
|
|
82
|
+
| `ANTHROPIC_AUTH_TOKEN` / `ANTHROPIC_BASE_URL` / `ANTHROPIC_MODEL` | ❌ | 用 Anthropic 兼容代理时的认证与模型配置;设置 `ANTHROPIC_AUTH_TOKEN` 后可不设置 `ANTHROPIC_API_KEY` |
|
|
83
|
+
| `BENCH_LLM_JUDGE` | ❌ | 设为 `1` 启用 LLM-as-judge 评分 |
|
|
84
|
+
| `BENCH_JUDGE_MODEL` | ❌ | Judge 模型;启用 `BENCH_LLM_JUDGE=1` 时必须显式设置,不能复用主模型 |
|
|
85
|
+
| `BENCH_JUDGE_AUTH_TOKEN` / `BENCH_JUDGE_BASE_URL` | ❌ | Judge 专用 Anthropic 兼容代理认证与 URL;两者都设置时优先通过 Anthropic Messages HTTP 直接调用 judge |
|
|
86
|
+
| `BENCH_JUDGE_API_KEY` | ❌ | Judge 专用 Anthropic API key;如果设置 `BENCH_JUDGE_AUTH_TOKEN`,优先使用 auth token |
|
|
87
|
+
| `LANGSMITH_API_KEY` | ❌ | 仅 LangSmith 套件需要 |
|
|
88
|
+
| `LANGSMITH_PROJECT` / `LANGSMITH_TRACING` | ❌ | LangSmith 套件项目名与追踪开关;Claude Code 插件变量会从这组配置自动派生 |
|
|
89
|
+
| `TRACE_TO_LANGSMITH` | ❌ | 兼容旧配置的 LangSmith 追踪开关 |
|
|
90
|
+
| `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY` | Langfuse | `--suite langfuse` 运行模式的项目凭据;只从当前进程读取,不写入报告或 manifest |
|
|
91
|
+
| `LANGFUSE_BASE_URL` / `LANGFUSE_TRACING_ENVIRONMENT` | ❌ | Langfuse 区域地址与 trace 环境标签 |
|
|
92
|
+
| `COMET_EVAL_ADAPTERS_DIR` | ❌ | 覆盖自定义 Agent 适配器注册目录;默认是用户目录下的 `~/.comet/eval/adapters` |
|
|
93
|
+
|
|
94
|
+
`*` 主任务至少需要 `BENCH_API_KEY` 或所选 Agent 的原生凭据之一。Claude Code、Codex 和 CodeBuddy 支持 `BENCH_BASE_URL`;Qoder 使用其官方支持的认证和服务地址配置。
|
|
95
|
+
|
|
96
|
+
安全边界:Eval 不会把 API key 写入发布包、manifest、报告或 Skill 工作区。Docker 运行 Agent 时,Codex、Qoder、CodeBuddy 使用容器内独立的配置根;Codex 的 `config.toml` 只引用 `env_key`,CodeBuddy 的 `settings.json` 只使用 `apiKeyHelper`,实际密钥只存在于本次容器进程环境中,运行结束后随临时配置目录销毁。
|
|
97
|
+
|
|
98
|
+
`comet eval` 默认使用 `claude-code`。可以通过 CLI 或 manifest 选择执行 Agent;CLI 优先于
|
|
99
|
+
manifest,未指定时回退到 Claude Code:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
comet eval ./generated-skill/comet/eval.yaml --agent codex
|
|
103
|
+
comet eval ./generated-skill/comet/eval.yaml --agent qoder
|
|
104
|
+
comet eval ./generated-skill/comet/eval.yaml --agent codebuddy
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
CodeBuddy 的自定义模型使用其 OpenAI-compatible 配置:`CODEBUDDY_API_KEY`、
|
|
108
|
+
`CODEBUDDY_BASE_URL` 和 provider 接受的 `CODEBUDDY_MODEL`,也可以直接用 `--model`
|
|
109
|
+
覆盖当前运行的模型。宿主机的 `~/.codebuddy/models.json` 只用于注册模型;Eval 不挂载它,
|
|
110
|
+
而是在容器内生成临时 `settings.json`,通过 `apiKeyHelper` 从进程环境读取凭据。不要把
|
|
111
|
+
Claude Code 的 Anthropic 地址或 Claude 专用模型 ID 直接复用给 CodeBuddy。
|
|
112
|
+
|
|
113
|
+
manifest 可声明默认 Agent:
|
|
114
|
+
|
|
115
|
+
```yaml
|
|
116
|
+
execution:
|
|
117
|
+
agent: codex
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
四种 Agent 共用 subject、auto_user simulator 和可选 Judge 的执行/报告契约;缺失的 token、成本或
|
|
121
|
+
上下文遥测会在报告中保留为 `N/A`,而不是用估算值补齐。
|
|
122
|
+
|
|
123
|
+
### Langfuse 套件
|
|
124
|
+
|
|
125
|
+
使用 `--suite langfuse` 时,local runner 的任务、Agent、Docker、validator 和本地报告保持不变,额外把每个 task/treatment 的核心结果写入 Langfuse,并在结束时 flush:
|
|
126
|
+
|
|
127
|
+
```bash
|
|
128
|
+
LANGFUSE_PUBLIC_KEY=pk-lf-... LANGFUSE_SECRET_KEY=sk-lf-... \
|
|
129
|
+
comet eval ./generated-skill/comet/eval.yaml --suite langfuse --html
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
`comet eval` 会自动选择 Langfuse optional extra。Claude Code 和 Codex 会在隔离的 `.comet/eval/langfuse/plugins/` 缓存中准备带完整性元数据的固定版本官方插件;Qoder 和 CodeBuddy 使用项目级 Stop hook 读取 JSONL transcript,转换失败时只降级详细轨迹,不影响本地评分和核心 trace。`--collect --suite langfuse` 不初始化 SDK、不联网、不下载插件。
|
|
133
|
+
|
|
134
|
+
## 快速开始
|
|
135
|
+
|
|
136
|
+
### 运行内置任务
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
# 运行单个任务 + treatment
|
|
140
|
+
uv run pytest local/tests/tasks/test_tasks.py --task=generic-skill-smoke --treatment=CONTROL -v
|
|
141
|
+
|
|
142
|
+
# 运行 Comet 工作流任务
|
|
143
|
+
uv run pytest local/tests/tasks/test_tasks.py --task=comet-full-workflow --treatment=COMET_FULL_040_BETA -v
|
|
144
|
+
|
|
145
|
+
# 运行所有任务
|
|
146
|
+
uv run pytest local/tests/tasks/test_tasks.py -v
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
### 报告中的去噪口径
|
|
150
|
+
|
|
151
|
+
Eval 报告同时保留 raw set 和 analysis set。Raw set 包含所有 run,方便追查原始 stdout/stderr、events、reports 和 artifacts;analysis set 默认排除明确环境噪声,用于 headline 指标、pass@k/pass^k、成本统计、图表和 verdict。
|
|
152
|
+
|
|
153
|
+
- `included`:可解释实验信号,进入主统计。真实 workflow、model、task 或 validator 失败仍属于这一类。
|
|
154
|
+
- `flagged`:可疑 harness/task 噪声,进入主统计,但报告会提示它可能影响结论。
|
|
155
|
+
- `excluded`:明确环境或运行器噪声,不进入主统计,例如 API timeout、rate limit、认证/网络失败、Docker/container failure 或外层 runner timeout。
|
|
156
|
+
|
|
157
|
+
如果关键 treatment 的 clean data 不足,报告会输出 `Insufficient clean data` 或 `Inconclusive due to data quality`,而不是给出误导性的胜负结论。
|
|
158
|
+
|
|
159
|
+
### 独立评估任意 Skill
|
|
160
|
+
|
|
161
|
+
独立评估不依赖 `comet-any`。只要有一个 Skill 目录,就可以直接运行;没有
|
|
162
|
+
`comet/eval.yaml` 时,普通运行会自动生成并缓存 2–4 个任务:
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
comet eval ./my-skill
|
|
166
|
+
comet eval ./my-skill --agent codebuddy --model subject-model
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
主模型和 LLM-as-judge 是两套独立配置。Judge 只有在显式启用后才会运行,至少需要自己的
|
|
170
|
+
`BENCH_JUDGE_MODEL` 和 `BENCH_JUDGE_API_KEY`/`BENCH_JUDGE_AUTH_TOKEN`;它可以使用不同的
|
|
171
|
+
Agent、模型和 API 地址:
|
|
172
|
+
|
|
173
|
+
```bash
|
|
174
|
+
BENCH_LLM_JUDGE=1 \
|
|
175
|
+
BENCH_JUDGE_MODEL=judge-model \
|
|
176
|
+
BENCH_JUDGE_BASE_URL=https://judge.example/v1 \
|
|
177
|
+
BENCH_JUDGE_API_KEY=... \
|
|
178
|
+
comet eval ./my-skill --model subject-model --base-url https://subject.example/v1
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
也可以全部通过命令行或 manifest 配置:
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
comet eval ./my-skill \
|
|
185
|
+
--agent codebuddy --model subject-model --base-url https://subject.example/v1 \
|
|
186
|
+
--judge-agent claude-code --judge-model judge-model \
|
|
187
|
+
--judge-base-url https://judge.example/v1
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
如果只想检查任务、配置和快照,不执行 Agent、Docker、插件或网络请求,使用:
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
comet eval ./my-skill --collect
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
### 扩展自定义 Agent
|
|
197
|
+
|
|
198
|
+
非预定义的 Agent 不需要修改 Comet 源码,也不需要把适配器放进 npm 包。用户在自己的配置目录
|
|
199
|
+
中显式注册即可,Eval 不会仅因为某个可执行文件在 `PATH` 中就自动发现它:
|
|
200
|
+
|
|
201
|
+
| 系统 | 默认适配器目录 |
|
|
202
|
+
| --- | --- |
|
|
203
|
+
| macOS/Linux | `~/.comet/eval/adapters/<agent-id>/adapter.yaml` |
|
|
204
|
+
| Windows | `%USERPROFILE%\\.comet\\eval\\adapters\\<agent-id>\\adapter.yaml` |
|
|
205
|
+
|
|
206
|
+
也可以在用户级 `~/.comet/eval/.env`(Windows 为 `%USERPROFILE%\\.comet\\eval\\.env`)中设置
|
|
207
|
+
`COMET_EVAL_ADAPTERS_DIR`,把整个适配器注册表放到其他位置。`<agent-id>` 必须是小写字母开头、
|
|
208
|
+
只包含小写字母/数字/连字符、长度 2–32 的标识符。
|
|
209
|
+
|
|
210
|
+
创建 `<agent-id>/adapter.yaml`,例如:
|
|
211
|
+
|
|
212
|
+
```yaml
|
|
213
|
+
apiVersion: comet.eval.agent/v1alpha1
|
|
214
|
+
kind: EvalAgentAdapter
|
|
215
|
+
metadata: { id: my-agent, version: 1.0.0 }
|
|
216
|
+
runtime:
|
|
217
|
+
executable: my-agent
|
|
218
|
+
install: { kind: npm, package: my-agent, version: latest }
|
|
219
|
+
credentials: [MY_AGENT_API_KEY]
|
|
220
|
+
modelEnv: MY_AGENT_MODEL
|
|
221
|
+
baseUrlEnv: MY_AGENT_BASE_URL
|
|
222
|
+
capabilities:
|
|
223
|
+
singleTurn: true
|
|
224
|
+
resume: true
|
|
225
|
+
structuredEvents: true
|
|
226
|
+
telemetry: false
|
|
227
|
+
skillInvocationEvidence: true
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
字段含义:
|
|
231
|
+
|
|
232
|
+
- `runtime.executable` 是容器内实际启动的命令;npm/pip 包必须暴露这个可执行入口。
|
|
233
|
+
`runtime.install.kind` 可为 `npm`、`pip` 或 `none`;使用 `npm`/`pip` 时 Eval 会在本次 Docker
|
|
234
|
+
镜像构建中安装对应包,`none` 表示命令已在基础镜像中准备好。
|
|
235
|
+
- `credentials` 只能填写最多两个环境变量名,不要填写密钥值。主 Agent 运行时,只有这些声明的
|
|
236
|
+
环境变量会被转发到容器;Judge 运行时,`BENCH_JUDGE_API_KEY` 映射到第一个凭据名,
|
|
237
|
+
`BENCH_JUDGE_AUTH_TOKEN` 映射到第二个凭据名(如果声明了第二个)。
|
|
238
|
+
- `modelEnv` 和 `baseUrlEnv` 是可选的环境变量名。主 Agent 可通过这些变量,或 CLI 的
|
|
239
|
+
`--model`/`--base-url`,或 manifest 的 `execution.model`/`execution.baseUrl` 配置;自定义 Agent
|
|
240
|
+
不会自动把 `BENCH_MODEL`、`BENCH_BASE_URL` 或 `BENCH_API_KEY` 映射到自定义变量。
|
|
241
|
+
- `capabilities` 描述适配器的真实能力。`resume` 用于 `auto_user` 多轮交互;`structuredEvents`
|
|
242
|
+
要求 CLI 输出 JSON Lines;`skillInvocationEvidence` 表示输出中能提供明确的 Skill 调用事件;
|
|
243
|
+
`telemetry: false` 时报告中的 token/cost 可以是 `N/A`。运行真实评估至少需要
|
|
244
|
+
`singleTurn`、`resume`、`structuredEvents` 和 `skillInvocationEvidence` 为 `true`。
|
|
245
|
+
|
|
246
|
+
自定义 CLI 必须接受 Eval 生成的调用形态:
|
|
247
|
+
|
|
248
|
+
```text
|
|
249
|
+
<executable> -p "<prompt>" --output-format stream-json --model <model> [--resume <session-id>]
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
并把每条结构化事件写到 stdout。若要让 rubric 认定 Skill 被调用,输出中需要有类似下面的 JSONL
|
|
253
|
+
事件(普通文件产物或路径推断不算自定义 Agent 的调用证据):
|
|
254
|
+
|
|
255
|
+
```json
|
|
256
|
+
{"type":"skill_invocation","skill":"my-skill"}
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
在用户级 `.env` 中填写适配器声明的变量,然后运行:
|
|
260
|
+
|
|
261
|
+
```dotenv
|
|
262
|
+
BENCH_EVAL_AGENT=my-agent
|
|
263
|
+
MY_AGENT_API_KEY=your-subject-key
|
|
264
|
+
MY_AGENT_MODEL=your-model
|
|
265
|
+
MY_AGENT_BASE_URL=https://your-provider.example/v1
|
|
266
|
+
|
|
267
|
+
# 如果让它作为独立 Judge:
|
|
268
|
+
# BENCH_LLM_JUDGE=1
|
|
269
|
+
# BENCH_JUDGE_AGENT=my-agent
|
|
270
|
+
# BENCH_JUDGE_API_KEY=your-judge-key
|
|
271
|
+
# BENCH_JUDGE_MODEL=your-judge-model
|
|
272
|
+
# BENCH_JUDGE_BASE_URL=https://your-judge-provider.example/v1
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
```bash
|
|
276
|
+
comet eval ./my-skill --collect --agent my-agent
|
|
277
|
+
comet eval ./my-skill --quick --agent my-agent --model your-model
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
`--collect` 会校验适配器和配置但不会启动 Agent、Docker 或联网;确认通过后再运行真实评估。
|
|
281
|
+
如果你的 CLI 不能兼容上述参数和 JSONL 事件契约,需要提供一个 wrapper CLI,将它转换为该契约;
|
|
282
|
+
当前 `adapter.yaml` 不支持自定义参数模板或直接挂载宿主机 Agent 配置目录。适配器文件可以分发给
|
|
283
|
+
团队成员,但真实凭据必须留在每个人自己的用户级 `.env` 或当前 shell 中。
|
|
284
|
+
|
|
285
|
+
### 评估任意 Skill
|
|
286
|
+
|
|
287
|
+
```bash
|
|
288
|
+
uv run pytest local/tests/tasks/test_tasks.py \
|
|
289
|
+
--task=generic-skill-smoke \
|
|
290
|
+
--skill-path=/path/to/my-skill \
|
|
291
|
+
--skill-name=my-skill \
|
|
292
|
+
--profile=generic -v
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
### 评估 /comet-any 生成的包
|
|
296
|
+
|
|
297
|
+
```bash
|
|
298
|
+
uv run pytest local/tests/tasks/test_tasks.py \
|
|
299
|
+
--eval-manifest=/path/to/my-skill/comet/eval.yaml -v
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
如果 manifest 没有 `evaluation.tasks` 或 `recommendedTasks`,普通运行会基于 Skill 的受限快照自动生成 2–4 个任务,并按 Agent、profile、交互配置和快照 hash 缓存到项目的 `.comet/eval/generated/`。缓存 manifest 会写入生成 hash 和元数据,方便复现;`--collect` 只接受已有缓存,不会在收集阶段调用生成 Agent。需要快速冒烟时显式使用:
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
comet eval ./my-skill --quick
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
## 容器中的 Agent 实验工作区
|
|
309
|
+
|
|
310
|
+
每次实验都会先在宿主机创建一个临时隔离目录,然后把它挂载到所选 Agent 的 Docker 容器的 `/workspace`。Agent 的当前工作目录就是 `/workspace`;双 agent 交互模式还会只读挂载 scaffold shell 到 `/opt/scaffold-shell`。
|
|
311
|
+
|
|
312
|
+
典型结构如下(以 `COMET_FULL_040_BETA` 为例,展示 0.4.0 beta 的 Comet Skill + 完整 dependency snapshot;实际任务产物会随 task/treatment 变化):
|
|
313
|
+
|
|
314
|
+
```text
|
|
315
|
+
/workspace # 所选 Agent 的当前工作目录;宿主机临时隔离目录挂载点
|
|
316
|
+
|-- Dockerfile # 当前 task 的 Docker 构建文件,来自 task/environment/
|
|
317
|
+
|-- requirements.txt # Python 依赖文件;仅 task 提供时存在
|
|
318
|
+
|-- package.json # Node 依赖文件;仅 task 提供时存在
|
|
319
|
+
|-- package-lock.json # Node 锁文件;仅 task 提供时存在
|
|
320
|
+
|-- tsconfig.json # TypeScript 配置;仅 task 提供时存在
|
|
321
|
+
|-- CLAUDE.md # Claude 项目级指令;其他 Agent 使用 AGENTS.md
|
|
322
|
+
|-- .eval-task-prompt.txt # auto_user 模式下的任务提示词临时文件,运行后清理
|
|
323
|
+
|-- .eval-simulator-prompt.txt # auto_user 模式下的用户模拟器提示词临时文件,运行后清理
|
|
324
|
+
|-- .claude / .agents / .qoder / .codebuddy # 所选 Agent 的本地配置和 Skill 安装目录
|
|
325
|
+
| |-- CLAUDE.md # Claude 专用项目指令副本
|
|
326
|
+
| |-- settings.json # Claude Code settings;包含 Comet PreToolUse hook
|
|
327
|
+
| `-- skills # 当前 treatment 注入的完整 Skill 包
|
|
328
|
+
| |-- comet # 040 beta 主 /comet Skill;039 baseline 也安装到这个 canonical 名称
|
|
329
|
+
| | |-- SKILL.md # 主 Skill 入口说明
|
|
330
|
+
| | |-- rules # Comet 软规则目录,完整复制自 benchmark snapshot
|
|
331
|
+
| | | `-- comet-phase-guard.md # phase guard 规则;040 beta 示例为中文规则文件
|
|
332
|
+
| | |-- reference # Comet reference 文档目录,供 Skill 运行时读取
|
|
333
|
+
| | | |-- auto-transition.md # 自动阶段推进协议
|
|
334
|
+
| | | |-- comet-yaml-fields.md # .comet.yaml 字段说明
|
|
335
|
+
| | | |-- context-recovery.md # 上下文恢复协议
|
|
336
|
+
| | | |-- debug-gate.md # 异常调试协议
|
|
337
|
+
| | | |-- decision-point.md # 用户决策点协议
|
|
338
|
+
| | | |-- dirty-worktree.md # dirty worktree 处理协议
|
|
339
|
+
| | | |-- file-structure.md # Comet 产物目录约定
|
|
340
|
+
| | | |-- intent-frame.md # 040 beta intent routing 字段说明
|
|
341
|
+
| | | |-- scripts.md # 运行脚本说明
|
|
342
|
+
| | | `-- subagent-dispatch.md # 子代理分发协议
|
|
343
|
+
| | |-- runtime # 040 beta runtime 元数据目录
|
|
344
|
+
| | | `-- classic # classic runtime package metadata
|
|
345
|
+
| | | |-- checks.yaml # comet skill check 的检查定义
|
|
346
|
+
| | | |-- guardrails.yaml # classic runtime guardrail 配置
|
|
347
|
+
| | | `-- skill.yaml # classic runtime Skill metadata
|
|
348
|
+
| | `-- scripts # 040 beta Node 脚本目录;完整复制自 benchmark snapshot
|
|
349
|
+
| | |-- comet-hook-guard.mjs # 040 beta Claude Code PreToolUse hook 入口
|
|
350
|
+
| | |-- comet-archive.mjs # archive 阶段脚本
|
|
351
|
+
| | |-- comet-env.mjs # runtime 环境解析脚本
|
|
352
|
+
| | |-- comet-guard.mjs # phase guard 脚本
|
|
353
|
+
| | |-- comet-handoff.mjs # handoff 写入脚本
|
|
354
|
+
| | |-- comet-intent.mjs # intent routing 脚本
|
|
355
|
+
| | |-- comet-state.mjs # .comet.yaml 状态机脚本
|
|
356
|
+
| | `-- comet-yaml-validate.mjs # .comet.yaml schema 校验脚本
|
|
357
|
+
| |-- comet-open # 040 beta /comet-open 阶段 Skill;完整包复制
|
|
358
|
+
| |-- comet-design # 040 beta /comet-design 阶段 Skill;完整包复制
|
|
359
|
+
| |-- comet-build # 040 beta /comet-build 阶段 Skill;完整包复制
|
|
360
|
+
| |-- comet-verify # 040 beta /comet-verify 阶段 Skill;完整包复制
|
|
361
|
+
| |-- comet-archive # 040 beta /comet-archive 阶段 Skill;完整包复制
|
|
362
|
+
| |-- comet-hotfix # 040 beta /comet-hotfix 工作流 Skill;完整包复制
|
|
363
|
+
| |-- comet-tweak # 040 beta /comet-tweak 工作流 Skill;完整包复制
|
|
364
|
+
| |-- openspec-apply-change # OpenSpec dependency Skill;完整包复制
|
|
365
|
+
| |-- openspec-archive-change # OpenSpec dependency Skill;完整包复制
|
|
366
|
+
| |-- openspec-bulk-archive-change # OpenSpec dependency Skill;完整包复制
|
|
367
|
+
| |-- openspec-continue-change # OpenSpec dependency Skill;完整包复制
|
|
368
|
+
| |-- openspec-explore # OpenSpec dependency Skill;完整包复制
|
|
369
|
+
| |-- openspec-ff-change # OpenSpec dependency Skill;完整包复制
|
|
370
|
+
| |-- openspec-new-change # OpenSpec dependency Skill;完整包复制
|
|
371
|
+
| |-- openspec-onboard # OpenSpec dependency Skill;完整包复制
|
|
372
|
+
| |-- openspec-propose # OpenSpec dependency Skill;完整包复制
|
|
373
|
+
| |-- openspec-sync-specs # OpenSpec dependency Skill;完整包复制
|
|
374
|
+
| |-- openspec-verify-change # OpenSpec dependency Skill;完整包复制
|
|
375
|
+
| |-- brainstorming # Superpowers dependency Skill;带 visual companion 和 scripts
|
|
376
|
+
| | |-- SKILL.md # brainstorming Skill 入口
|
|
377
|
+
| | |-- scripts # brainstorming 辅助脚本目录
|
|
378
|
+
| | `-- visual-companion.md # brainstorming visual companion 说明
|
|
379
|
+
| |-- dispatching-parallel-agents # Superpowers dependency Skill;完整包复制
|
|
380
|
+
| |-- executing-plans # Superpowers dependency Skill;完整包复制
|
|
381
|
+
| |-- finishing-a-development-branch # Superpowers dependency Skill;完整包复制
|
|
382
|
+
| |-- receiving-code-review # Superpowers dependency Skill;完整包复制
|
|
383
|
+
| |-- requesting-code-review # Superpowers dependency Skill;完整包复制
|
|
384
|
+
| |-- subagent-driven-development # Superpowers dependency Skill;带 scripts 和 reviewer prompt
|
|
385
|
+
| | |-- SKILL.md # subagent-driven-development Skill 入口
|
|
386
|
+
| | |-- scripts # subagent workspace/review 辅助脚本目录
|
|
387
|
+
| | `-- task-reviewer-prompt.md # task reviewer prompt 文件
|
|
388
|
+
| |-- systematic-debugging # Superpowers dependency Skill;完整包复制
|
|
389
|
+
| |-- test-driven-development # Superpowers dependency Skill;完整包复制
|
|
390
|
+
| |-- using-git-worktrees # Superpowers dependency Skill;完整包复制
|
|
391
|
+
| |-- verification-before-completion # Superpowers dependency Skill;完整包复制
|
|
392
|
+
| |-- writing-plans # Superpowers dependency Skill;完整包复制
|
|
393
|
+
| `-- writing-skills # Superpowers dependency Skill;带 examples 等支持文件
|
|
394
|
+
| |-- SKILL.md # writing-skills Skill 入口
|
|
395
|
+
| `-- examples # writing-skills 示例目录
|
|
396
|
+
|-- .comet # Comet 项目配置/运行状态目录,由 workflow 创建或更新
|
|
397
|
+
| `-- config.yaml # Comet 项目配置;由 init/update 或 task 环境提供
|
|
398
|
+
|-- openspec # OpenSpec 工作区目录,由 workflow 创建或更新
|
|
399
|
+
| `-- changes # OpenSpec change 产物目录
|
|
400
|
+
|-- docs # repo 文档目录;workflow 可能写入设计/计划/证据
|
|
401
|
+
| `-- superpowers # Superpowers 设计、计划、审查证据目录
|
|
402
|
+
|-- _test_results.json # validation 脚本输出的结构化结果
|
|
403
|
+
`-- <task files and agent artifacts> # task 初始代码和 agent 运行中创建的产物
|
|
404
|
+
|
|
405
|
+
/opt/scaffold-shell # 双 agent loop 模式下只读挂载的 runner shell 脚本目录
|
|
406
|
+
|-- run-claude-loop.sh # subject agent 和 simulator agent 的循环驱动脚本
|
|
407
|
+
|-- docker.sh # Docker build/run 辅助脚本
|
|
408
|
+
|-- setup.sh # workspace setup 辅助脚本
|
|
409
|
+
`-- common.sh # shell 公共函数
|
|
410
|
+
```
|
|
411
|
+
|
|
412
|
+
说明:
|
|
413
|
+
|
|
414
|
+
- `Dockerfile`、依赖文件和任务初始代码来自当前 task 的 `environment/`,例如 `stats.py`、`test_stats.py`。
|
|
415
|
+
- `CLAUDE.md` 只在需要项目级指令时写入;`comet-workflow` profile 会写入强制先触发 `/comet` 的 benchmark 指令。
|
|
416
|
+
- `.eval-task-prompt.txt` 和 `.eval-simulator-prompt.txt` 只在 `auto_user` 双 agent 模式运行期间存在,运行结束后会清理。
|
|
417
|
+
- `.claude/settings.json` 会为 Comet baseline 配置 Claude Code `PreToolUse` hook。`COMET_FULL_040_BETA` 指向 `comet-hook-guard.mjs`;`COMET_FULL_039` 指向 `comet-hook-guard.sh`。
|
|
418
|
+
- `.claude/skills/*` 是完整 Skill 包复制,不只是 `SKILL.md`;OpenSpec 和 Superpowers 依赖的 `scripts/`、`examples/`、prompt 文件和其他随包文件都会保留。
|
|
419
|
+
- `COMET_FULL_039` 的目录形状与上面一致,但 `comet` 和 `comet-*` 来自 `039-release/*` 快照,主脚本是 `.sh`:`comet-hook-guard.sh`、`comet-state.sh`、`comet-guard.sh`、`comet-handoff.sh`、`comet-archive.sh`、`comet-yaml-validate.sh`。OpenSpec / Superpowers dependency snapshot 与 040 baseline 相同。
|
|
420
|
+
- `openspec/`、`.comet/`、`docs/superpowers/`、任务代码修改和其他产物由 agent 在运行过程中创建或更新。
|
|
421
|
+
- LangSmith 轨迹追踪由官方 `langsmith-tracing` Claude Code 插件负责(不再使用手写 Stop hook);用户主要配置 `LANGSMITH_*`,`TRACE_TO_LANGSMITH` / `CC_LANGSMITH_*` 会自动派生。启用完整 Claude Code trajectory 时,插件预构建目录会以 `/opt/langsmith-cc-plugin` 只读挂载进容器,详见 `langsmith/README.md`。
|
|
422
|
+
- `/opt/scaffold-shell` 只在双 agent loop 运行时挂载,提供容器内调用 Claude 的循环驱动脚本。
|
|
423
|
+
|
|
424
|
+
## 定义自己的 Task
|
|
425
|
+
|
|
426
|
+
除了把任务注册到 `eval/local/tasks/`,项目也可以在自己的 `comet/eval.yaml` 中声明 inline task 或引用 Skill 包内的 task package。项目任务会和 `recommendedTasks` 合并;同名任务会直接拒绝,避免覆盖内置任务。
|
|
427
|
+
|
|
428
|
+
```yaml
|
|
429
|
+
evaluation:
|
|
430
|
+
tasks:
|
|
431
|
+
- name: writes-summary
|
|
432
|
+
prompt: Create summary.md with a short result.
|
|
433
|
+
workspace: fixtures/summary
|
|
434
|
+
expect:
|
|
435
|
+
files: [summary.md]
|
|
436
|
+
contains:
|
|
437
|
+
summary.md: ["# Summary"]
|
|
438
|
+
commands:
|
|
439
|
+
- run: test -s summary.md
|
|
440
|
+
timeout: 30
|
|
441
|
+
rubric:
|
|
442
|
+
- The summary is specific and actionable.
|
|
443
|
+
- source: tasks/advanced
|
|
444
|
+
```
|
|
445
|
+
|
|
446
|
+
`workspace`、`source` 和所有期望产物都必须留在 Skill 包/任务工作区内;`files`、`contains`、`json` 和 `commands` 检查在隔离 Docker 工作区中执行。`--collect` 只做解析和发现,不运行 Agent 或用户命令。
|
|
447
|
+
|
|
448
|
+
每个任务是 `local/tasks/` 下的一个目录,包含三个部分:
|
|
449
|
+
|
|
450
|
+
```text
|
|
451
|
+
local/tasks/my-task/
|
|
452
|
+
├── task.toml # 任务配置
|
|
453
|
+
├── instruction.md # 给 agent 的指令(支持 {run_id} 等模板变量)
|
|
454
|
+
├── environment/
|
|
455
|
+
│ └── Dockerfile # 验证环境(提供运行时和测试依赖)
|
|
456
|
+
└── validation/
|
|
457
|
+
└── test_my_task.py # 验证脚本(在 Docker 内运行)
|
|
458
|
+
```
|
|
459
|
+
|
|
460
|
+
### 1. 编写 `task.toml`
|
|
461
|
+
|
|
462
|
+
```toml
|
|
463
|
+
[metadata]
|
|
464
|
+
name = "my-task"
|
|
465
|
+
description = "任务描述"
|
|
466
|
+
difficulty = "medium" # easy / medium / hard
|
|
467
|
+
category = "generic" # generic 或 comet
|
|
468
|
+
default_treatments = ["CONTROL"]
|
|
469
|
+
|
|
470
|
+
[environment]
|
|
471
|
+
description = "Docker 环境描述"
|
|
472
|
+
dockerfile = "environment/Dockerfile"
|
|
473
|
+
timeout_sec = 600 # Docker 执行超时(秒)
|
|
474
|
+
|
|
475
|
+
[validation]
|
|
476
|
+
test_scripts = ["test_my_task.py"] # Docker 内运行的验证脚本
|
|
477
|
+
target_artifacts = ["result.md"] # 预期产物(存在性检查)
|
|
478
|
+
timeout = 120 # 验证脚本超时(秒)
|
|
479
|
+
|
|
480
|
+
[evaluation]
|
|
481
|
+
profile = "generic" # 评估 profile
|
|
482
|
+
expected_artifacts = ["result.md"] # rubric 检查的产物
|
|
483
|
+
required_skills = ["my-skill"] # 必须调用的 Skill
|
|
484
|
+
require_skill_invocation = true # 未调用时产生硬失败
|
|
485
|
+
|
|
486
|
+
# 自定义 rubric 标准(传给 LLM judge,可选)
|
|
487
|
+
rubric_criteria = [
|
|
488
|
+
"输出包含错误处理",
|
|
489
|
+
"代码遵循项目命名规范",
|
|
490
|
+
]
|
|
491
|
+
|
|
492
|
+
[interaction]
|
|
493
|
+
mode = "none" # none(单轮)或 auto_user(多轮模拟)
|
|
494
|
+
max_turns = 12 # 最大轮次
|
|
495
|
+
```
|
|
496
|
+
|
|
497
|
+
### 2. 编写 `instruction.md`
|
|
498
|
+
|
|
499
|
+
这是给 agent 的任务指令,支持 `{run_id}` 模板变量:
|
|
500
|
+
|
|
501
|
+
```markdown
|
|
502
|
+
在当前工作区创建一个文件 `result.md`。
|
|
503
|
+
|
|
504
|
+
要求:
|
|
505
|
+
|
|
506
|
+
- 包含标题 `# My Task Result`
|
|
507
|
+
- 描述你的实现思路
|
|
508
|
+
- 包含至少三个要点
|
|
509
|
+
```
|
|
510
|
+
|
|
511
|
+
### 3. 编写验证脚本
|
|
512
|
+
|
|
513
|
+
验证脚本在 Docker 内运行,结果写入 `_test_results.json`:
|
|
514
|
+
|
|
515
|
+
```python
|
|
516
|
+
from pathlib import Path
|
|
517
|
+
from scaffold.python.validation.core import write_test_results
|
|
518
|
+
|
|
519
|
+
def main():
|
|
520
|
+
passed = []
|
|
521
|
+
failed = []
|
|
522
|
+
|
|
523
|
+
# 检查产物是否存在
|
|
524
|
+
result = Path("result.md")
|
|
525
|
+
if not result.exists():
|
|
526
|
+
write_test_results({"passed": [], "failed": ["result.md missing"]})
|
|
527
|
+
return
|
|
528
|
+
|
|
529
|
+
text = result.read_text(encoding="utf-8")
|
|
530
|
+
|
|
531
|
+
# 检查内容
|
|
532
|
+
if "# My Task Result" in text:
|
|
533
|
+
passed.append("heading present")
|
|
534
|
+
else:
|
|
535
|
+
failed.append("heading missing")
|
|
536
|
+
|
|
537
|
+
write_test_results({"passed": passed, "failed": failed})
|
|
538
|
+
|
|
539
|
+
if __name__ == "__main__":
|
|
540
|
+
main()
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
### 4. 注册任务
|
|
544
|
+
|
|
545
|
+
在 `local/tasks/index.yaml` 中添加:
|
|
546
|
+
|
|
547
|
+
```yaml
|
|
548
|
+
tasks:
|
|
549
|
+
- name: my-task
|
|
550
|
+
category: generic
|
|
551
|
+
default_treatments:
|
|
552
|
+
- CONTROL
|
|
553
|
+
description: 我的自定义任务
|
|
554
|
+
```
|
|
555
|
+
|
|
556
|
+
## Rubric 评估体系
|
|
557
|
+
|
|
558
|
+
### 评估 Profile
|
|
559
|
+
|
|
560
|
+
框架通过 profile 决定使用哪套 rubric:
|
|
561
|
+
|
|
562
|
+
| Profile | 用途 | 维度数 | LLM Judge |
|
|
563
|
+
| ----------------- | ----------------- | ------ | ------------------------- |
|
|
564
|
+
| `generic` | 任意 Skill | 7 | ✅(`BENCH_LLM_JUDGE=1`) |
|
|
565
|
+
| `comet-workflow` | Comet 工作流 | 9 | ✅ |
|
|
566
|
+
| `authoring-skill` | /comet-any 生成包 | 11 | 继承 generic |
|
|
567
|
+
|
|
568
|
+
Comet 类任务(`category = "comet"` 或名称以 `comet-` 开头)自动推断为 `comet-workflow`。
|
|
569
|
+
|
|
570
|
+
### 通用 Rubric(7 维度)
|
|
571
|
+
|
|
572
|
+
用于评估任意 Skill 的执行质量:
|
|
573
|
+
|
|
574
|
+
| 维度 | 权重 | 说明 |
|
|
575
|
+
| ------------------------ | ---- | ------------------------------------------------------ |
|
|
576
|
+
| `completion` | 2.0 | 验证脚本通过率 |
|
|
577
|
+
| `skill_invocation` | 1.0 | 必须调用的 Skill 是否被调用(未配置时为 N/A) |
|
|
578
|
+
| `artifact_presence` | 1.0 | 预期产物是否存在(支持 glob,未配置时为 N/A) |
|
|
579
|
+
| `instruction_following` | 1.0 | 是否有约束违反 |
|
|
580
|
+
| `interaction_compliance` | 0.8 | 多轮交互是否在轮次限制内 |
|
|
581
|
+
| `efficiency` | 0.7 | 轮次 ≤ 80、工具调用 ≤ 150、时长 ≤ 600s |
|
|
582
|
+
| `safety_boundary` | 1.2 | 无危险命令(`rm -rf`、`git reset --hard`、`curl\|sh`) |
|
|
583
|
+
|
|
584
|
+
**N/A 维度处理**:当 `required_skills` 或 `expected_artifacts` 未配置时,对应维度标记为 N/A,不参与加权计算,避免虚高的中间分。
|
|
585
|
+
|
|
586
|
+
### LLM-as-judge(可选)
|
|
587
|
+
|
|
588
|
+
设置 `BENCH_LLM_JUDGE=1` 启用 LLM 评审。Judge 读取 agent 的 workspace 产物进行定性评分,但必须使用独立裁判配置:至少设置 `BENCH_JUDGE_MODEL`,如需走不同代理或账号,再设置 `BENCH_JUDGE_BASE_URL`、`BENCH_JUDGE_AUTH_TOKEN` 或 `BENCH_JUDGE_API_KEY`。
|
|
589
|
+
|
|
590
|
+
当配置了 `BENCH_JUDGE_BASE_URL` 和 `BENCH_JUDGE_AUTH_TOKEN` / `BENCH_JUDGE_API_KEY` 时,judge 会优先直接调用 Anthropic Messages HTTP(`/v1/messages`)。没有专用 judge endpoint 时,才回退到宿主机 `claude` CLI,并在子进程里清除继承来的主 `ANTHROPIC_*` provider 设置,再把 `BENCH_JUDGE_*` 映射为自己的 `ANTHROPIC_*` 环境变量。这样可以避免同一个模型既当选手又当裁判,也避免严格代理不兼容 Claude CLI 的额外请求参数。若启用了 `BENCH_LLM_JUDGE=1` 但未设置 `BENCH_JUDGE_MODEL`,报告会输出 `[RUBRIC-JUDGE] status: skipped - BENCH_JUDGE_MODEL is required when BENCH_LLM_JUDGE=1`,不会调用 judge。
|
|
591
|
+
|
|
592
|
+
**通用 judge 的三个标准维度**:
|
|
593
|
+
|
|
594
|
+
| 维度 | 评分标准 |
|
|
595
|
+
| ----------------------- | ------------------------------------------------------- |
|
|
596
|
+
| `task_completion` | 1.0 = 全部完成;0.5 = 部分完成;0.0 = 未完成 |
|
|
597
|
+
| `output_quality` | 1.0 = 结构化、完整、非平凡;0.5 = 浅层;0.0 = 空或 stub |
|
|
598
|
+
| `instruction_adherence` | 1.0 = 完全遵循;0.5 = 轻微偏差;0.0 = 严重违反 |
|
|
599
|
+
|
|
600
|
+
**自定义 rubric 标准**:在 `task.toml` 的 `[evaluation]` 中定义 `rubric_criteria`,这些标准会作为额外维度传给 LLM judge,输出为 `custom_0`、`custom_1` 等。
|
|
601
|
+
|
|
602
|
+
```toml
|
|
603
|
+
[evaluation]
|
|
604
|
+
rubric_criteria = [
|
|
605
|
+
"函数处理了边界情况(空输入、单元素)",
|
|
606
|
+
"错误信息用户友好且可操作",
|
|
607
|
+
]
|
|
608
|
+
```
|
|
609
|
+
|
|
610
|
+
### Comet Rubric(9 维度)
|
|
611
|
+
|
|
612
|
+
Comet 工作流专用,输出 9 个维度分数和一个 `weighted_score`。每个维度由若干二值检查组成,维度分数为 `通过检查数 / 总检查数`;最终 `weighted_score` 按下表权重加权平均。
|
|
613
|
+
|
|
614
|
+
| 维度 | 权重 | 评分信号 |
|
|
615
|
+
| --------------------------- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
616
|
+
| `completion` | 2.0 | 当前 task 的基础验证脚本通过率,例如功能修复、测试文件、工作流产物等 task-specific checks |
|
|
617
|
+
| `main_flow` | 1.5 | workflow 阶段证据是否齐全;`full` 检查 open → design → build → verify → archive,`hotfix` / `tweak` 检查 open → build → verify → archive |
|
|
618
|
+
| `gate_guard` | 1.5 | 是否使用 Comet 阶段守卫和状态推进能力,包括 `comet-guard`、状态 transition、`--apply` |
|
|
619
|
+
| `artifact_quality` | 1.2 | OpenSpec / Superpowers / 测试产物是否有实质内容;`full` 要求 proposal、深度 design、足够 tasks 和测试断言,`hotfix` / `tweak` 使用预设路径的轻量产物口径 |
|
|
620
|
+
| `skill_invocation` | 1.0 | 是否真实调用主 `comet`、嵌套 Comet 阶段 Skill、OpenSpec 依赖 Skill、Superpowers 依赖 Skill,并保持入口先于子 Skill |
|
|
621
|
+
| `spec_drift` | 1.0 | 如果写入 delta spec,是否在归档前通过 OpenSpec 同步/归档语义合并;未涉及 delta spec 时记为 N/A 式满分 |
|
|
622
|
+
| `decision_point_compliance` | 1.0 | 到达阻塞决策点时是否先询问用户;`full` 会检查 build/verify 等状态决策,`hotfix` / `tweak` 只检查预设路径中的升级、验证失败、归档重开等阻塞决策 |
|
|
623
|
+
| `recovery_resilience` | 1.0 | 是否保留可恢复状态,例如 `.comet/checkpoint.json`、`run-state.json`、`state-events.jsonl`、`trajectory.jsonl`、handoff/context、干净 pending 状态 |
|
|
624
|
+
| `efficiency` | 0.8 | 运行成本是否在阈值内:turns ≤ 80、工具调用 ≤ 150、时长 ≤ 600s |
|
|
625
|
+
|
|
626
|
+
额外规则:
|
|
627
|
+
|
|
628
|
+
- `skills_invoked` 只来自 Claude Code 事件中的真实 Skill 工具调用和 Skill 文件加载,不根据产物反推。
|
|
629
|
+
- `opsx:*` 旧 OpenSpec Skill 名会归一化为 `openspec-*`,避免同一调用在报告中重复出现。
|
|
630
|
+
- 如果 `comet-workflow` 未调用主 `comet`,会产生硬失败;如果调用了主 `comet` 但没有调用嵌套 Comet / OpenSpec / Superpowers 依赖 Skill,也会产生 workflow contract failure。
|
|
631
|
+
- `weighted_score` 只聚合 9 个 Comet rubric 维度;LLM judge 启用后会额外输出 `[RUBRIC-JUDGE]` 信息,不替代规则分。
|
|
632
|
+
|
|
633
|
+
## Treatment 配置
|
|
634
|
+
|
|
635
|
+
Treatment 定义注入哪些 Skill。位于 `local/treatments/`:
|
|
636
|
+
|
|
637
|
+
```yaml
|
|
638
|
+
# local/treatments/common/control.yaml
|
|
639
|
+
CONTROL:
|
|
640
|
+
description: '无 Skill 基线'
|
|
641
|
+
skills: []
|
|
642
|
+
|
|
643
|
+
# local/treatments/comet/comet_full_040_beta.yaml
|
|
644
|
+
COMET_FULL_040_BETA:
|
|
645
|
+
description: '完整 Comet 工作流'
|
|
646
|
+
skills:
|
|
647
|
+
- name: comet
|
|
648
|
+
skill: 040-beta/comet
|
|
649
|
+
variant: all
|
|
650
|
+
base: benchmarks
|
|
651
|
+
- name: comet-open
|
|
652
|
+
skill: 040-beta/comet-open
|
|
653
|
+
variant: all
|
|
654
|
+
base: benchmarks
|
|
655
|
+
```
|
|
656
|
+
|
|
657
|
+
## 运行测试
|
|
658
|
+
|
|
659
|
+
```bash
|
|
660
|
+
# 仅运行 scaffold 单元测试(快速,不需要 Docker)
|
|
661
|
+
cd eval
|
|
662
|
+
uv run pytest local/tests/scaffold/ -v
|
|
663
|
+
|
|
664
|
+
# 运行完整任务测试(需要 Docker + Claude CLI)
|
|
665
|
+
uv run pytest local/tests/tasks/test_tasks.py -v
|
|
666
|
+
|
|
667
|
+
# 重复运行 N 次(统计通过率分布)
|
|
668
|
+
uv run pytest local/tests/tasks/test_tasks.py --count=3 -v
|
|
669
|
+
|
|
670
|
+
# 并行运行
|
|
671
|
+
uv run pytest local/tests/tasks/test_tasks.py -n 4 -v
|
|
672
|
+
```
|
|
673
|
+
|
|
674
|
+
## 核心指标
|
|
675
|
+
|
|
676
|
+
### pass@k — 能力上限
|
|
677
|
+
|
|
678
|
+
> 给定 k 次独立尝试,**至少有一次成功**的概率。
|
|
679
|
+
|
|
680
|
+
使用 HumanEval 的无偏估计器:
|
|
681
|
+
|
|
682
|
+
```
|
|
683
|
+
pass@k = 1 - C(n-c, k) / C(n, k)
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
其中 `n` = 总运行次数,`c` = 成功次数。当 `n - c < k` 时(不可能连续抽到 k 次失败),`pass@k = 1.0`。
|
|
687
|
+
|
|
688
|
+
**解读**:pass@k 衡量"agent 能不能做到"。pass@1 就是单次通过率;pass@5 接近 1.0 意味着多次尝试中几乎总能找到一个成功的。
|
|
689
|
+
|
|
690
|
+
### pass^k — 可靠性下限
|
|
691
|
+
|
|
692
|
+
> 给定 k 次独立尝试,**全部成功**的概率。
|
|
693
|
+
|
|
694
|
+
```
|
|
695
|
+
pass^k = 1.0 (if c == n, 即所有运行都通过)
|
|
696
|
+
pass^k = 0.0 (otherwise, 观测样本下限)
|
|
697
|
+
```
|
|
698
|
+
|
|
699
|
+
**解读**:pass^k 衡量"agent 每次都能做到吗"。只有当所有运行都通过时才为 1.0,否则为 0.0。这是可靠性的一致性下界。
|
|
700
|
+
|
|
701
|
+
### pass@k − pass^k — 不稳定性间隙
|
|
702
|
+
|
|
703
|
+
两者的差值量化了**不稳定性**:高 pass@k + 低 pass^k = "能做到,但不能指望每次都做到"。对于用户反复运行的工作流 Skill,这个间隙是关键质量信号。
|
|
704
|
+
|
|
705
|
+
### 示例
|
|
706
|
+
|
|
707
|
+
| Treatment | pass@1 | pass@5 | pass^1 | pass^5 | pass/fail |
|
|
708
|
+
| ------------------- | ------ | ------ | ------ | ------ | --------- |
|
|
709
|
+
| CONTROL | 0.40 | 0.90 | 0 | 0 | 2/5 |
|
|
710
|
+
| COMET_FULL_040_BETA | 0.80 | 1.00 | 0 | 0 | 4/5 |
|
|
711
|
+
|
|
712
|
+
- CONTROL:40% 单次通过率,5 次尝试有 90% 概率成功至少一次,但无法保证每次都成功
|
|
713
|
+
- COMET_FULL_040_BETA:80% 单次通过率,5 次尝试几乎必然成功,但仍有波动
|
|
714
|
+
|
|
715
|
+
### 运行多次获取 pass@k
|
|
716
|
+
|
|
717
|
+
```bash
|
|
718
|
+
# 重复运行 5 次以获得 pass@5 / pass^5
|
|
719
|
+
uv run pytest local/tests/tasks/test_tasks.py --count=5 -v
|
|
720
|
+
```
|
|
721
|
+
|
|
722
|
+
报告自动生成在 `comparison_report.md` 中,包含 pass@k / pass^k 表格。
|
|
723
|
+
|
|
724
|
+
## 查看结果
|
|
725
|
+
|
|
726
|
+
评估结果写入 `local/logs/experiments/`,包含:
|
|
727
|
+
|
|
728
|
+
- `summary.md` — 汇总表(各 treatment 的维度分数)
|
|
729
|
+
- `comparison_report.md` — 基线对比报告(含 pass@k / pass^k 指标)
|
|
730
|
+
- 每次运行的详细日志和产物
|
|
731
|
+
|
|
732
|
+
默认输出 Markdown,可通过 `--report-config` 或 `COMET_EVAL_REPORT_CONFIG` 启用 HTML:
|
|
733
|
+
|
|
734
|
+
```json
|
|
735
|
+
{
|
|
736
|
+
"report_outputs": {
|
|
737
|
+
"markdown": true,
|
|
738
|
+
"html": true
|
|
739
|
+
}
|
|
740
|
+
}
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
## CLI 参数速查
|
|
744
|
+
|
|
745
|
+
| 参数 | 说明 |
|
|
746
|
+
| ------------------------ | ------------------------------ |
|
|
747
|
+
| `--task=<name>` | 指定任务 |
|
|
748
|
+
| `--treatment=<name>` | 指定 treatment(逗号分隔多个) |
|
|
749
|
+
| `--profile=<name>` | 覆盖评估 profile |
|
|
750
|
+
| `--agent=<id>` | 主 Agent;可用内置 Agent 或已安装的自定义适配器 |
|
|
751
|
+
| `--model=<name>` | 主模型覆盖 |
|
|
752
|
+
| `--base-url=<url>` | 主模型 API 地址覆盖 |
|
|
753
|
+
| `--judge-agent=<id>` | 独立 Judge Agent |
|
|
754
|
+
| `--judge-model=<name>` | 独立 Judge 模型 |
|
|
755
|
+
| `--judge-base-url=<url>` | 独立 Judge API 地址 |
|
|
756
|
+
| `--collect` | 只解析/发现,不执行工作负载 |
|
|
757
|
+
| `--skill-path=<path>` | 注入本地 Skill 路径 |
|
|
758
|
+
| `--skill-name=<name>` | 注入的 Skill 名称 |
|
|
759
|
+
| `--count=<n>` | 重复运行次数 |
|
|
760
|
+
| `--eval-manifest=<path>` | 使用生成的 eval.yaml |
|
|
761
|
+
| `--report-config=<path>` | 报告输出配置 |
|