@rpamis/comet 0.4.0-beta.3 → 0.4.0-beta.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +15 -1
- package/assets/manifest.json +2 -1
- package/assets/skills/comet/SKILL.md +54 -28
- package/assets/skills/comet/reference/auto-transition.md +1 -1
- package/assets/skills/comet/reference/comet-yaml-fields.md +11 -6
- package/assets/skills/comet/reference/context-recovery.md +12 -2
- package/assets/skills/comet/reference/decision-point.md +11 -0
- package/assets/skills/comet/reference/scripts.md +47 -15
- package/assets/skills/comet/reference/subagent-dispatch.md +6 -5
- package/assets/skills/comet/rules/comet-phase-guard.en.md +31 -28
- package/assets/skills/comet/rules/comet-phase-guard.md +31 -29
- package/assets/skills/comet/scripts/comet-resume-probe.mjs +3 -0
- package/assets/skills/comet/scripts/comet-runtime.mjs +1568 -476
- package/assets/skills/comet-any/SKILL.md +1 -1
- package/assets/skills/comet-any/reference/authored-zone-example.md +10 -5
- package/assets/skills/comet-any/reference/subagents/pause-points-author.md +14 -11
- package/assets/skills/comet-any/reference/subagents/skill-reviewer.md +2 -0
- package/assets/skills/comet-any/reference/subagents/workflow-entry-author.md +5 -4
- package/assets/skills/comet-archive/SKILL.md +52 -18
- package/assets/skills/comet-build/SKILL.md +64 -45
- package/assets/skills/comet-design/SKILL.md +24 -22
- package/assets/skills/comet-hotfix/SKILL.md +37 -34
- package/assets/skills/comet-open/SKILL.md +106 -57
- package/assets/skills/comet-tweak/SKILL.md +23 -18
- package/assets/skills/comet-verify/SKILL.md +57 -72
- package/assets/skills-zh/comet/SKILL.md +54 -28
- package/assets/skills-zh/comet/reference/auto-transition.md +1 -1
- package/assets/skills-zh/comet/reference/comet-yaml-fields.md +11 -6
- package/assets/skills-zh/comet/reference/context-recovery.md +12 -2
- package/assets/skills-zh/comet/reference/decision-point.md +11 -0
- package/assets/skills-zh/comet/reference/scripts.md +47 -15
- package/assets/skills-zh/comet/reference/subagent-dispatch.md +6 -5
- package/assets/skills-zh/comet-any/SKILL.md +1 -1
- package/assets/skills-zh/comet-any/reference/authored-zone-example.md +10 -5
- package/assets/skills-zh/comet-any/reference/subagents/pause-points-author.md +14 -12
- package/assets/skills-zh/comet-any/reference/subagents/skill-reviewer.md +2 -0
- package/assets/skills-zh/comet-any/reference/subagents/workflow-entry-author.md +5 -4
- package/assets/skills-zh/comet-archive/SKILL.md +52 -18
- package/assets/skills-zh/comet-build/SKILL.md +64 -45
- package/assets/skills-zh/comet-design/SKILL.md +24 -22
- package/assets/skills-zh/comet-hotfix/SKILL.md +36 -33
- package/assets/skills-zh/comet-open/SKILL.md +105 -56
- package/assets/skills-zh/comet-tweak/SKILL.md +22 -17
- package/assets/skills-zh/comet-verify/SKILL.md +57 -72
- package/dist/app/cli/comet-banner.d.ts +25 -0
- package/dist/app/cli/comet-banner.d.ts.map +1 -0
- package/dist/app/cli/comet-banner.js +265 -0
- package/dist/app/cli/comet-banner.js.map +1 -0
- package/dist/app/cli/index.js +36 -2
- package/dist/app/cli/index.js.map +1 -1
- package/dist/app/commands/classic.d.ts +4 -0
- package/dist/app/commands/classic.d.ts.map +1 -0
- package/dist/app/commands/classic.js +11 -0
- package/dist/app/commands/classic.js.map +1 -0
- package/dist/app/commands/doctor.d.ts.map +1 -1
- package/dist/app/commands/doctor.js +99 -37
- package/dist/app/commands/doctor.js.map +1 -1
- package/dist/app/commands/eval.d.ts.map +1 -1
- package/dist/app/commands/eval.js +57 -20
- package/dist/app/commands/eval.js.map +1 -1
- package/dist/app/commands/i18n.d.ts +1 -1
- package/dist/app/commands/i18n.d.ts.map +1 -1
- package/dist/app/commands/i18n.js +12 -4
- package/dist/app/commands/i18n.js.map +1 -1
- package/dist/app/commands/init.d.ts +1 -1
- package/dist/app/commands/init.d.ts.map +1 -1
- package/dist/app/commands/init.js +72 -25
- package/dist/app/commands/init.js.map +1 -1
- package/dist/app/commands/project-scope-selection.d.ts +13 -0
- package/dist/app/commands/project-scope-selection.d.ts.map +1 -0
- package/dist/app/commands/project-scope-selection.js +33 -0
- package/dist/app/commands/project-scope-selection.js.map +1 -0
- package/dist/app/commands/resume-probe.d.ts +11 -0
- package/dist/app/commands/resume-probe.d.ts.map +1 -0
- package/dist/app/commands/resume-probe.js +63 -0
- package/dist/app/commands/resume-probe.js.map +1 -0
- package/dist/app/commands/status.d.ts +31 -0
- package/dist/app/commands/status.d.ts.map +1 -1
- package/dist/app/commands/status.js +109 -20
- package/dist/app/commands/status.js.map +1 -1
- package/dist/app/commands/uninstall.d.ts +5 -0
- package/dist/app/commands/uninstall.d.ts.map +1 -1
- package/dist/app/commands/uninstall.js +264 -53
- package/dist/app/commands/uninstall.js.map +1 -1
- package/dist/app/commands/update.d.ts +6 -0
- package/dist/app/commands/update.d.ts.map +1 -1
- package/dist/app/commands/update.js +558 -86
- package/dist/app/commands/update.js.map +1 -1
- package/dist/config/repository-layout.json +4 -2
- package/dist/domains/bundle/bundle-platform.d.ts +1 -0
- package/dist/domains/bundle/bundle-platform.d.ts.map +1 -1
- package/dist/domains/bundle/bundle-platform.js +9 -3
- package/dist/domains/bundle/bundle-platform.js.map +1 -1
- package/dist/domains/bundle/eval-manifest-runtime.d.ts +6 -0
- package/dist/domains/bundle/eval-manifest-runtime.d.ts.map +1 -0
- package/dist/domains/bundle/eval-manifest-runtime.js +58 -0
- package/dist/domains/bundle/eval-manifest-runtime.js.map +1 -0
- package/dist/domains/bundle/types.d.ts +1 -1
- package/dist/domains/bundle/types.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-archive.d.ts +1 -0
- package/dist/domains/comet-classic/classic-archive.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-archive.js +35 -38
- package/dist/domains/comet-classic/classic-archive.js.map +1 -1
- package/dist/domains/comet-classic/classic-cli.d.ts +1 -1
- package/dist/domains/comet-classic/classic-cli.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-cli.js +3 -0
- package/dist/domains/comet-classic/classic-cli.js.map +1 -1
- package/dist/domains/comet-classic/classic-command-checks.d.ts +20 -0
- package/dist/domains/comet-classic/classic-command-checks.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-command-checks.js +97 -0
- package/dist/domains/comet-classic/classic-command-checks.js.map +1 -0
- package/dist/domains/comet-classic/classic-current-change.d.ts +19 -0
- package/dist/domains/comet-classic/classic-current-change.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-current-change.js +132 -0
- package/dist/domains/comet-classic/classic-current-change.js.map +1 -0
- package/dist/domains/comet-classic/classic-diagnostics.d.ts +2 -1
- package/dist/domains/comet-classic/classic-diagnostics.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-diagnostics.js +54 -1
- package/dist/domains/comet-classic/classic-diagnostics.js.map +1 -1
- package/dist/domains/comet-classic/classic-evidence.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-evidence.js +4 -0
- package/dist/domains/comet-classic/classic-evidence.js.map +1 -1
- package/dist/domains/comet-classic/classic-guard.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-guard.js +83 -80
- package/dist/domains/comet-classic/classic-guard.js.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-hook-guard.js +179 -42
- package/dist/domains/comet-classic/classic-hook-guard.js.map +1 -1
- package/dist/domains/comet-classic/classic-project-config.d.ts +16 -0
- package/dist/domains/comet-classic/classic-project-config.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-project-config.js +40 -0
- package/dist/domains/comet-classic/classic-project-config.js.map +1 -0
- package/dist/domains/comet-classic/classic-resolver.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-resolver.js +4 -4
- package/dist/domains/comet-classic/classic-resolver.js.map +1 -1
- package/dist/domains/comet-classic/classic-resume-probe-command.d.ts +3 -0
- package/dist/domains/comet-classic/classic-resume-probe-command.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe-command.js +58 -0
- package/dist/domains/comet-classic/classic-resume-probe-command.js.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.d.ts +2 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.js +4 -0
- package/dist/domains/comet-classic/classic-resume-probe-entry.js.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe.d.ts +29 -0
- package/dist/domains/comet-classic/classic-resume-probe.d.ts.map +1 -0
- package/dist/domains/comet-classic/classic-resume-probe.js +386 -0
- package/dist/domains/comet-classic/classic-resume-probe.js.map +1 -0
- package/dist/domains/comet-classic/classic-runtime-run.d.ts +2 -1
- package/dist/domains/comet-classic/classic-runtime-run.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-runtime-run.js +38 -0
- package/dist/domains/comet-classic/classic-runtime-run.js.map +1 -1
- package/dist/domains/comet-classic/classic-state-command.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-state-command.js +150 -25
- package/dist/domains/comet-classic/classic-state-command.js.map +1 -1
- package/dist/domains/comet-classic/classic-state.d.ts +5 -2
- package/dist/domains/comet-classic/classic-state.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-state.js +17 -1
- package/dist/domains/comet-classic/classic-state.js.map +1 -1
- package/dist/domains/comet-classic/classic-store.d.ts +4 -1
- package/dist/domains/comet-classic/classic-store.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-store.js +4 -3
- package/dist/domains/comet-classic/classic-store.js.map +1 -1
- package/dist/domains/comet-classic/classic-transitions.d.ts +1 -1
- package/dist/domains/comet-classic/classic-transitions.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-transitions.js +36 -3
- package/dist/domains/comet-classic/classic-transitions.js.map +1 -1
- package/dist/domains/comet-classic/classic-validate-command.d.ts.map +1 -1
- package/dist/domains/comet-classic/classic-validate-command.js +8 -1
- package/dist/domains/comet-classic/classic-validate-command.js.map +1 -1
- package/dist/domains/comet-classic/index.d.ts +3 -0
- package/dist/domains/comet-classic/index.d.ts.map +1 -1
- package/dist/domains/comet-classic/index.js +3 -0
- package/dist/domains/comet-classic/index.js.map +1 -1
- package/dist/domains/dashboard/collector.js +10 -2
- package/dist/domains/dashboard/collector.js.map +1 -1
- package/dist/domains/dashboard/web/assets/_commonjsHelpers-CqkleIqs.js +1 -0
- package/dist/domains/dashboard/web/assets/arc-TGVUG2WN.js +1 -0
- package/dist/domains/dashboard/web/assets/architectureDiagram-3BPJPVTR-ASHmiHoP.js +36 -0
- package/dist/domains/dashboard/web/assets/blockDiagram-GPEHLZMM-CwASxBKe.js +132 -0
- package/dist/domains/dashboard/web/assets/c4Diagram-AAUBKEIU-BU8z_WNk.js +10 -0
- package/dist/domains/dashboard/web/assets/channel-kp6Ng2JJ.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-2J33WTMH-CBGlUx4_.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-4BX2VUAB-CS-iGcFz.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-55IACEB6-BUtCVJat.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-727SXJPM-C94tj_pf.js +206 -0
- package/dist/domains/dashboard/web/assets/chunk-AQP2D5EJ-BFOKcpdG.js +231 -0
- package/dist/domains/dashboard/web/assets/chunk-FMBD7UC4-Dmx8B-st.js +15 -0
- package/dist/domains/dashboard/web/assets/chunk-ND2GUHAM-DYV00CNn.js +1 -0
- package/dist/domains/dashboard/web/assets/chunk-QZHKN3VN-DMJJyQnw.js +1 -0
- package/dist/domains/dashboard/web/assets/classDiagram-4FO5ZUOK-yBDjgMY_.js +1 -0
- package/dist/domains/dashboard/web/assets/classDiagram-v2-Q7XG4LA2-yBDjgMY_.js +1 -0
- package/dist/domains/dashboard/web/assets/cose-bilkent-S5V4N54A-P4nE2rea.js +1 -0
- package/dist/domains/dashboard/web/assets/cynefin-VYW2F7L2-C-R1HXDv.js +166 -0
- package/dist/domains/dashboard/web/assets/cytoscape.esm-D3_iZ_3b.js +321 -0
- package/dist/domains/dashboard/web/assets/dagre-BM42HDAG-BH8d-C9z.js +4 -0
- package/dist/domains/dashboard/web/assets/defaultLocale-DX6XiGOO.js +1 -0
- package/dist/domains/dashboard/web/assets/diagram-2AECGRRQ-DpXTgQIH.js +43 -0
- package/dist/domains/dashboard/web/assets/diagram-5GNKFQAL-DSiFJzSw.js +10 -0
- package/dist/domains/dashboard/web/assets/diagram-KO2AKTUF-DULMkben.js +3 -0
- package/dist/domains/dashboard/web/assets/diagram-LMA3HP47-Pi8m_NO9.js +24 -0
- package/dist/domains/dashboard/web/assets/diagram-OG6HWLK6-BSe6bCWm.js +24 -0
- package/dist/domains/dashboard/web/assets/erDiagram-TEJ5UH35-zb-FEcbq.js +85 -0
- package/dist/domains/dashboard/web/assets/flowDiagram-I6XJVG4X-Dr-Gk0z2.js +162 -0
- package/dist/domains/dashboard/web/assets/ganttDiagram-6RSMTGT7-qs6Zb2A6.js +292 -0
- package/dist/domains/dashboard/web/assets/gitGraphDiagram-PVQCEYII-BQ314orT.js +106 -0
- package/dist/domains/dashboard/web/assets/graph--OzhPTMs.js +1 -0
- package/dist/domains/dashboard/web/assets/index-BLAfkNTu.css +1 -0
- package/dist/domains/dashboard/web/assets/index-DlSYK56b.js +139 -0
- package/dist/domains/dashboard/web/assets/index-DtLUOcAW.js +24 -0
- package/dist/domains/dashboard/web/assets/index-QO0z6F4w.js +10 -0
- package/dist/domains/dashboard/web/assets/infoDiagram-5YYISTIA-O0bQp98A.js +2 -0
- package/dist/domains/dashboard/web/assets/init-Gi6I4Gst.js +1 -0
- package/dist/domains/dashboard/web/assets/ishikawaDiagram-YF4QCWOH-BPC7wbKh.js +70 -0
- package/dist/domains/dashboard/web/assets/journeyDiagram-JHISSGLW-ohViOd4b.js +139 -0
- package/dist/domains/dashboard/web/assets/kanban-definition-UN3LZRKU-CHleNdbG.js +89 -0
- package/dist/domains/dashboard/web/assets/katex-HP8lGamR.js +257 -0
- package/dist/domains/dashboard/web/assets/layout-SsrduOYp.js +1 -0
- package/dist/domains/dashboard/web/assets/linear-CyqiW--Q.js +1 -0
- package/dist/domains/dashboard/web/assets/marked.esm-DECB3bxB.js +64 -0
- package/dist/domains/dashboard/web/assets/mermaid.core-qS9XvtaK.js +301 -0
- package/dist/domains/dashboard/web/assets/mindmap-definition-RKZ34NQL-oaGkqO9l.js +96 -0
- package/dist/domains/dashboard/web/assets/ordinal-Cboi1Yqb.js +1 -0
- package/dist/domains/dashboard/web/assets/pieDiagram-4H26LBE5-CPcB-AFh.js +30 -0
- package/dist/domains/dashboard/web/assets/purify.es-VaSPOPhr.js +3 -0
- package/dist/domains/dashboard/web/assets/quadrantDiagram-W4KKPZXB-LFy52Qtj.js +7 -0
- package/dist/domains/dashboard/web/assets/requirementDiagram-4Y6WPE33-BdjeaYP_.js +84 -0
- package/dist/domains/dashboard/web/assets/sankeyDiagram-5OEKKPKP-DDJ4styL.js +40 -0
- package/dist/domains/dashboard/web/assets/sequenceDiagram-3UESZ5HK-BL5bFNoz.js +162 -0
- package/dist/domains/dashboard/web/assets/stateDiagram-AJRCARHV-BNUfikui.js +1 -0
- package/dist/domains/dashboard/web/assets/stateDiagram-v2-BHNVJYJU-DGrsmWts.js +1 -0
- package/dist/domains/dashboard/web/assets/timeline-definition-PNZ67QCA--ps2uu6L.js +120 -0
- package/dist/domains/dashboard/web/assets/vennDiagram-CIIHVFJN-C2q8q26c.js +34 -0
- package/dist/domains/dashboard/web/assets/wardleyDiagram-YWT4CUSO-BUH9gXqd.js +78 -0
- package/dist/domains/dashboard/web/assets/xychartDiagram-2RQKCTM6-B-DielEy.js +7 -0
- package/dist/domains/dashboard/web/index.html +16 -16
- package/dist/domains/engine/types.d.ts +1 -1
- package/dist/domains/engine/types.d.ts.map +1 -1
- package/dist/domains/factory/package.d.ts.map +1 -1
- package/dist/domains/factory/package.js +20 -6
- package/dist/domains/factory/package.js.map +1 -1
- package/dist/domains/integrations/openspec.d.ts +4 -1
- package/dist/domains/integrations/openspec.d.ts.map +1 -1
- package/dist/domains/integrations/openspec.js +62 -10
- package/dist/domains/integrations/openspec.js.map +1 -1
- package/dist/domains/skill/find.d.ts.map +1 -1
- package/dist/domains/skill/find.js +1 -6
- package/dist/domains/skill/find.js.map +1 -1
- package/dist/domains/skill/json-object.d.ts +13 -0
- package/dist/domains/skill/json-object.d.ts.map +1 -0
- package/dist/domains/skill/json-object.js +24 -0
- package/dist/domains/skill/json-object.js.map +1 -0
- package/dist/domains/skill/managed-markdown.d.ts +14 -0
- package/dist/domains/skill/managed-markdown.d.ts.map +1 -0
- package/dist/domains/skill/managed-markdown.js +126 -0
- package/dist/domains/skill/managed-markdown.js.map +1 -0
- package/dist/domains/skill/platform-inspect.d.ts +9 -0
- package/dist/domains/skill/platform-inspect.d.ts.map +1 -0
- package/dist/domains/skill/platform-inspect.js +149 -0
- package/dist/domains/skill/platform-inspect.js.map +1 -0
- package/dist/domains/skill/platform-install.d.ts +20 -9
- package/dist/domains/skill/platform-install.d.ts.map +1 -1
- package/dist/domains/skill/platform-install.js +356 -153
- package/dist/domains/skill/platform-install.js.map +1 -1
- package/dist/domains/skill/project-instructions.d.ts +22 -0
- package/dist/domains/skill/project-instructions.d.ts.map +1 -0
- package/dist/domains/skill/project-instructions.js +61 -0
- package/dist/domains/skill/project-instructions.js.map +1 -0
- package/dist/domains/skill/uninstall.d.ts +3 -1
- package/dist/domains/skill/uninstall.d.ts.map +1 -1
- package/dist/domains/skill/uninstall.js +259 -149
- package/dist/domains/skill/uninstall.js.map +1 -1
- package/dist/platform/fs/file-system.d.ts +3 -3
- package/dist/platform/fs/file-system.d.ts.map +1 -1
- package/dist/platform/fs/file-system.js +21 -11
- package/dist/platform/fs/file-system.js.map +1 -1
- package/dist/platform/install/detect.d.ts +2 -1
- package/dist/platform/install/detect.d.ts.map +1 -1
- package/dist/platform/install/detect.js +10 -1
- package/dist/platform/install/detect.js.map +1 -1
- package/dist/platform/install/platforms.d.ts +9 -1
- package/dist/platform/install/platforms.d.ts.map +1 -1
- package/dist/platform/install/platforms.js +23 -4
- package/dist/platform/install/platforms.js.map +1 -1
- package/dist/platform/install/project-registry.d.ts +38 -0
- package/dist/platform/install/project-registry.d.ts.map +1 -0
- package/dist/platform/install/project-registry.js +213 -0
- package/dist/platform/install/project-registry.js.map +1 -0
- package/dist/platform/install/skill-root-owner.d.ts +15 -0
- package/dist/platform/install/skill-root-owner.d.ts.map +1 -0
- package/dist/platform/install/skill-root-owner.js +68 -0
- package/dist/platform/install/skill-root-owner.js.map +1 -0
- package/eval/.env.example +52 -0
- package/eval/CLAUDE.md +43 -0
- package/eval/README.md +538 -0
- package/eval/langsmith/.env.example +16 -0
- package/eval/langsmith/README.md +93 -0
- package/eval/langsmith/skills/README.md +5 -0
- package/eval/langsmith/tasks/README.md +5 -0
- package/eval/langsmith/tests/conftest.py +187 -0
- package/eval/langsmith/tests/tasks/test_tasks.py +287 -0
- package/eval/langsmith/treatments/README.md +5 -0
- package/eval/local/README.md +80 -0
- package/eval/local/regression_baseline.json +13 -0
- package/eval/local/report-style-demo-assets/quality_cost.en.svg +41 -0
- package/eval/local/report-style-demo-assets/quality_cost.zh.svg +41 -0
- package/eval/local/report-style-demo-assets/rubric_delta.en.svg +56 -0
- package/eval/local/report-style-demo-assets/rubric_delta.zh.svg +56 -0
- package/eval/local/report-style-demo-assets/task_outcomes.en.svg +46 -0
- package/eval/local/report-style-demo-assets/task_outcomes.zh.svg +46 -0
- package/eval/local/report-style-demo.html +867 -0
- package/eval/local/scripts/compare_baselines.py +1018 -0
- package/eval/local/scripts/generate_report_style_demo_charts.py +322 -0
- package/eval/local/scripts/regression_check.py +171 -0
- package/eval/local/scripts/render_readme_benchmark_figures.py +416 -0
- package/eval/local/scripts/rescore_rubric.py +120 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/SKILL.md +282 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/auto-transition.md +27 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/comet-yaml-fields.md +68 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/context-recovery.md +35 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/debug-gate.md +17 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/decision-point.md +20 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/dirty-worktree.md +59 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/file-structure.md +28 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/subagent-dispatch.md +113 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/rules/comet-phase-guard.md +125 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-archive.sh +311 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-env.sh +110 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-guard.sh +778 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-handoff.sh +390 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-hook-guard.sh +336 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-state.sh +1338 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-yaml-validate.sh +225 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-archive/SKILL.md +100 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-build/SKILL.md +304 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-design/SKILL.md +263 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-hotfix/SKILL.md +204 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-open/SKILL.md +220 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-tweak/SKILL.md +180 -0
- package/eval/local/skills/benchmarks/039-release/comet-classic-039-verify/SKILL.md +234 -0
- package/eval/local/skills/benchmarks/040-beta/comet/SKILL.md +261 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/auto-transition.md +27 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/comet-yaml-fields.md +73 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/context-recovery.md +45 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/debug-gate.md +33 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/decision-point.md +31 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/dirty-worktree.md +59 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/file-structure.md +28 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/intent-frame.md +74 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/scripts.md +69 -0
- package/eval/local/skills/benchmarks/040-beta/comet/reference/subagent-dispatch.md +169 -0
- package/eval/local/skills/benchmarks/040-beta/comet/rules/comet-phase-guard.md +130 -0
- package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/checks.yaml +6 -0
- package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/guardrails.yaml +13 -0
- package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/skill.yaml +122 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-archive.mjs +9515 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-env.mjs +15 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-guard.mjs +10173 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-handoff.mjs +9491 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-hook-guard.mjs +9175 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-intent.mjs +404 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-state.mjs +9273 -0
- package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-yaml-validate.mjs +7627 -0
- package/eval/local/skills/benchmarks/040-beta/comet-archive/SKILL.md +110 -0
- package/eval/local/skills/benchmarks/040-beta/comet-build/SKILL.md +315 -0
- package/eval/local/skills/benchmarks/040-beta/comet-design/SKILL.md +260 -0
- package/eval/local/skills/benchmarks/040-beta/comet-hotfix/SKILL.md +190 -0
- package/eval/local/skills/benchmarks/040-beta/comet-open/SKILL.md +210 -0
- package/eval/local/skills/benchmarks/040-beta/comet-tweak/SKILL.md +183 -0
- package/eval/local/skills/benchmarks/040-beta/comet-verify/SKILL.md +230 -0
- package/eval/local/skills/benchmarks/dependency/claude-md/comet-workflow/CLAUDE.md +12 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-apply-change/SKILL.md +156 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-archive-change/SKILL.md +114 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-bulk-archive-change/SKILL.md +246 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-continue-change/SKILL.md +118 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-explore/SKILL.md +288 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-ff-change/SKILL.md +101 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-new-change/SKILL.md +74 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-onboard/SKILL.md +554 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-propose/SKILL.md +110 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-sync-specs/SKILL.md +138 -0
- package/eval/local/skills/benchmarks/dependency/openspec/openspec-verify-change/SKILL.md +168 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/SKILL.md +159 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/frame-template.html +213 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/helper.js +167 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/server.cjs +723 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/start-server.sh +209 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/stop-server.sh +120 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/spec-document-reviewer-prompt.md +49 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/visual-companion.md +291 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/dispatching-parallel-agents/SKILL.md +185 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/executing-plans/SKILL.md +70 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/finishing-a-development-branch/SKILL.md +241 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/receiving-code-review/SKILL.md +213 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/requesting-code-review/SKILL.md +103 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/requesting-code-review/code-reviewer.md +172 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/SKILL.md +418 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/implementer-prompt.md +139 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/review-package +44 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/sdd-workspace +22 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/task-brief +40 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/task-reviewer-prompt.md +188 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/CREATION-LOG.md +119 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/SKILL.md +296 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/condition-based-waiting-example.ts +158 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/condition-based-waiting.md +115 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/defense-in-depth.md +122 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/find-polluter.sh +63 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/root-cause-tracing.md +169 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-academic.md +14 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-1.md +58 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-2.md +68 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-3.md +69 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/test-driven-development/SKILL.md +371 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/test-driven-development/testing-anti-patterns.md +299 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/using-git-worktrees/SKILL.md +202 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/verification-before-completion/SKILL.md +139 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-plans/SKILL.md +174 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-plans/plan-document-reviewer-prompt.md +49 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/SKILL.md +689 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/anthropic-best-practices.md +1150 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/examples/CLAUDE_MD_TESTING.md +189 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/graphviz-conventions.dot +172 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/persuasion-principles.md +187 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/render-graphs.js +168 -0
- package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/testing-skills-with-subagents.md +384 -0
- package/eval/local/tasks/authoring-skill-smoke/environment/Dockerfile +2 -0
- package/eval/local/tasks/authoring-skill-smoke/instruction.md +4 -0
- package/eval/local/tasks/authoring-skill-smoke/task.toml +21 -0
- package/eval/local/tasks/authoring-skill-smoke/validation/test_authoring_skill_smoke.py +54 -0
- package/eval/local/tasks/comet-agent-memory-routing/environment/Dockerfile +28 -0
- package/eval/local/tasks/comet-agent-memory-routing/environment/agent_system.py +40 -0
- package/eval/local/tasks/comet-agent-memory-routing/environment/test_agent_system.py +29 -0
- package/eval/local/tasks/comet-agent-memory-routing/instruction.md +20 -0
- package/eval/local/tasks/comet-agent-memory-routing/task.toml +24 -0
- package/eval/local/tasks/comet-agent-memory-routing/validation/test_agent_memory_routing.py +24 -0
- package/eval/local/tasks/comet-api-cache-ttl/environment/Dockerfile +39 -0
- package/eval/local/tasks/comet-api-cache-ttl/environment/cache.py +39 -0
- package/eval/local/tasks/comet-api-cache-ttl/environment/test_cache.py +81 -0
- package/eval/local/tasks/comet-api-cache-ttl/instruction.md +33 -0
- package/eval/local/tasks/comet-api-cache-ttl/task.toml +24 -0
- package/eval/local/tasks/comet-api-cache-ttl/validation/test_api_cache_ttl.py +58 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/cli.py +13 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/counting.py +13 -0
- package/eval/local/tasks/comet-cross-file-refactor/environment/test_textkit.py +21 -0
- package/eval/local/tasks/comet-cross-file-refactor/instruction.md +16 -0
- package/eval/local/tasks/comet-cross-file-refactor/task.toml +20 -0
- package/eval/local/tasks/comet-cross-file-refactor/validation/test_cross_file_refactor.py +20 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/Dockerfile +28 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/app.py +8 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/settings_loader.py +14 -0
- package/eval/local/tasks/comet-dependency-confusion/environment/test_settings_service.py +19 -0
- package/eval/local/tasks/comet-dependency-confusion/instruction.md +18 -0
- package/eval/local/tasks/comet-dependency-confusion/task.toml +24 -0
- package/eval/local/tasks/comet-dependency-confusion/validation/test_dependency_confusion.py +24 -0
- package/eval/local/tasks/comet-fix-median/environment/Dockerfile +50 -0
- package/eval/local/tasks/comet-fix-median/environment/stats.py +54 -0
- package/eval/local/tasks/comet-fix-median/environment/test_stats.py +68 -0
- package/eval/local/tasks/comet-fix-median/instruction.md +25 -0
- package/eval/local/tasks/comet-fix-median/task.toml +24 -0
- package/eval/local/tasks/comet-fix-median/validation/test_fix_median.py +60 -0
- package/eval/local/tasks/comet-framework-selection/environment/Dockerfile +28 -0
- package/eval/local/tasks/comet-framework-selection/environment/architecture.py +12 -0
- package/eval/local/tasks/comet-framework-selection/environment/test_architecture.py +30 -0
- package/eval/local/tasks/comet-framework-selection/instruction.md +22 -0
- package/eval/local/tasks/comet-framework-selection/task.toml +24 -0
- package/eval/local/tasks/comet-framework-selection/validation/test_framework_selection.py +24 -0
- package/eval/local/tasks/comet-full-workflow/environment/Dockerfile +53 -0
- package/eval/local/tasks/comet-full-workflow/environment/test_wordcount.py +29 -0
- package/eval/local/tasks/comet-full-workflow/environment/wordcount.py +40 -0
- package/eval/local/tasks/comet-full-workflow/instruction.md +18 -0
- package/eval/local/tasks/comet-full-workflow/task.toml +24 -0
- package/eval/local/tasks/comet-full-workflow/validation/test_full_workflow.py +171 -0
- package/eval/local/tasks/comet-graph-execution-review/environment/Dockerfile +28 -0
- package/eval/local/tasks/comet-graph-execution-review/environment/pipeline.py +19 -0
- package/eval/local/tasks/comet-graph-execution-review/environment/test_pipeline.py +53 -0
- package/eval/local/tasks/comet-graph-execution-review/instruction.md +21 -0
- package/eval/local/tasks/comet-graph-execution-review/task.toml +24 -0
- package/eval/local/tasks/comet-graph-execution-review/validation/test_graph_execution_review.py +24 -0
- package/eval/local/tasks/comet-human-approval-flow/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-human-approval-flow/environment/approvals.py +17 -0
- package/eval/local/tasks/comet-human-approval-flow/environment/test_approvals.py +30 -0
- package/eval/local/tasks/comet-human-approval-flow/instruction.md +15 -0
- package/eval/local/tasks/comet-human-approval-flow/task.toml +20 -0
- package/eval/local/tasks/comet-human-approval-flow/validation/test_human_approval_flow.py +20 -0
- package/eval/local/tasks/comet-layered-streaming-fix/environment/Dockerfile +20 -0
- package/eval/local/tasks/comet-layered-streaming-fix/environment/chat_app.py +19 -0
- package/eval/local/tasks/comet-layered-streaming-fix/environment/test_chat_app.py +15 -0
- package/eval/local/tasks/comet-layered-streaming-fix/instruction.md +14 -0
- package/eval/local/tasks/comet-layered-streaming-fix/task.toml +20 -0
- package/eval/local/tasks/comet-layered-streaming-fix/validation/test_layered_streaming_fix.py +20 -0
- package/eval/local/tasks/comet-noise-distractor/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-noise-distractor/environment/distractor.md +4 -0
- package/eval/local/tasks/comet-noise-distractor/environment/invoice.py +5 -0
- package/eval/local/tasks/comet-noise-distractor/environment/test_invoice.py +18 -0
- package/eval/local/tasks/comet-noise-distractor/instruction.md +17 -0
- package/eval/local/tasks/comet-noise-distractor/task.toml +20 -0
- package/eval/local/tasks/comet-noise-distractor/validation/test_noise_distractor.py +20 -0
- package/eval/local/tasks/comet-observability-env-template/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-observability-env-template/environment/env_writer.py +17 -0
- package/eval/local/tasks/comet-observability-env-template/environment/test_env_writer.py +17 -0
- package/eval/local/tasks/comet-observability-env-template/instruction.md +20 -0
- package/eval/local/tasks/comet-observability-env-template/task.toml +20 -0
- package/eval/local/tasks/comet-observability-env-template/validation/test_observability_env_template.py +46 -0
- package/eval/local/tasks/comet-perf-dedupe/environment/Dockerfile +39 -0
- package/eval/local/tasks/comet-perf-dedupe/environment/dedupe.py +22 -0
- package/eval/local/tasks/comet-perf-dedupe/environment/test_dedupe.py +43 -0
- package/eval/local/tasks/comet-perf-dedupe/instruction.md +25 -0
- package/eval/local/tasks/comet-perf-dedupe/task.toml +24 -0
- package/eval/local/tasks/comet-perf-dedupe/validation/test_perf_dedupe.py +40 -0
- package/eval/local/tasks/comet-persistence-threading/environment/Dockerfile +15 -0
- package/eval/local/tasks/comet-persistence-threading/environment/session_store.py +15 -0
- package/eval/local/tasks/comet-persistence-threading/environment/test_session_store.py +21 -0
- package/eval/local/tasks/comet-persistence-threading/instruction.md +14 -0
- package/eval/local/tasks/comet-persistence-threading/task.toml +20 -0
- package/eval/local/tasks/comet-persistence-threading/validation/test_persistence_threading.py +20 -0
- package/eval/local/tasks/comet-refactor-counter/environment/Dockerfile +39 -0
- package/eval/local/tasks/comet-refactor-counter/environment/test_text_processor.py +57 -0
- package/eval/local/tasks/comet-refactor-counter/environment/text_processor.py +49 -0
- package/eval/local/tasks/comet-refactor-counter/instruction.md +31 -0
- package/eval/local/tasks/comet-refactor-counter/task.toml +24 -0
- package/eval/local/tasks/comet-refactor-counter/validation/test_refactor_counter.py +75 -0
- package/eval/local/tasks/comet-robust-config/environment/Dockerfile +39 -0
- package/eval/local/tasks/comet-robust-config/environment/config_loader.py +56 -0
- package/eval/local/tasks/comet-robust-config/environment/test_config_loader.py +72 -0
- package/eval/local/tasks/comet-robust-config/instruction.md +30 -0
- package/eval/local/tasks/comet-robust-config/task.toml +24 -0
- package/eval/local/tasks/comet-robust-config/validation/test_robust_config.py +37 -0
- package/eval/local/tasks/generic-skill-smoke/environment/Dockerfile +2 -0
- package/eval/local/tasks/generic-skill-smoke/instruction.md +7 -0
- package/eval/local/tasks/generic-skill-smoke/task.toml +21 -0
- package/eval/local/tasks/generic-skill-smoke/validation/test_generic_skill_smoke.py +36 -0
- package/eval/local/tasks/index.yaml +155 -0
- package/eval/local/tasks/workflow-overlay-contract/environment/Dockerfile +5 -0
- package/eval/local/tasks/workflow-overlay-contract/instruction.md +5 -0
- package/eval/local/tasks/workflow-overlay-contract/task.toml +21 -0
- package/eval/local/tasks/workflow-overlay-contract/validation/test_workflow_overlay_contract.py +235 -0
- package/eval/local/tasks/workflow-route-conformance/environment/Dockerfile +2 -0
- package/eval/local/tasks/workflow-route-conformance/instruction.md +5 -0
- package/eval/local/tasks/workflow-route-conformance/task.toml +21 -0
- package/eval/local/tasks/workflow-route-conformance/validation/test_workflow_route_conformance.py +84 -0
- package/eval/local/tests/conftest.py +1148 -0
- package/eval/local/tests/scaffold/test_attribution.py +48 -0
- package/eval/local/tests/scaffold/test_compare_baselines.py +999 -0
- package/eval/local/tests/scaffold/test_conftest_helpers.py +534 -0
- package/eval/local/tests/scaffold/test_evidence.py +15 -0
- package/eval/local/tests/scaffold/test_langsmith_conftest.py +244 -0
- package/eval/local/tests/scaffold/test_logging.py +243 -0
- package/eval/local/tests/scaffold/test_manifests.py +132 -0
- package/eval/local/tests/scaffold/test_profiles.py +866 -0
- package/eval/local/tests/scaffold/test_regression_check.py +19 -0
- package/eval/local/tests/scaffold/test_report_style_demo_charts.py +45 -0
- package/eval/local/tests/scaffold/test_sample_quality.py +211 -0
- package/eval/local/tests/scaffold/test_tasks.py +168 -0
- package/eval/local/tests/scaffold/test_treatments.py +244 -0
- package/eval/local/tests/scaffold/test_utils.py +139 -0
- package/eval/local/tests/tasks/test_tasks.py +445 -0
- package/eval/local/tests/tasks/test_validation_scripts.py +143 -0
- package/eval/local/treatments/comet/comet_full_039.yaml +141 -0
- package/eval/local/treatments/comet/comet_full_040_beta.yaml +140 -0
- package/eval/local/treatments/common/control.yaml +4 -0
- package/eval/pyproject.toml +41 -0
- package/eval/report-html-config.json +6 -0
- package/eval/scaffold/__init__.py +67 -0
- package/eval/scaffold/python/__init__.py +95 -0
- package/eval/scaffold/python/attribution.py +43 -0
- package/eval/scaffold/python/evidence.py +38 -0
- package/eval/scaffold/python/external_data_handler.py +18 -0
- package/eval/scaffold/python/generic_llm_judge.py +235 -0
- package/eval/scaffold/python/judge_config.py +168 -0
- package/eval/scaffold/python/llm_judge.py +191 -0
- package/eval/scaffold/python/logging.py +705 -0
- package/eval/scaffold/python/manifests.py +129 -0
- package/eval/scaffold/python/paper_charts.py +25 -0
- package/eval/scaffold/python/pass_at_k.py +107 -0
- package/eval/scaffold/python/paths.py +56 -0
- package/eval/scaffold/python/profiles.py +127 -0
- package/eval/scaffold/python/report_outputs.py +1391 -0
- package/eval/scaffold/python/sample_quality.py +339 -0
- package/eval/scaffold/python/schema.py +39 -0
- package/eval/scaffold/python/skill_parser.py +469 -0
- package/eval/scaffold/python/tasks.py +330 -0
- package/eval/scaffold/python/treatments.py +271 -0
- package/eval/scaffold/python/utils.py +366 -0
- package/eval/scaffold/python/validation/__init__.py +21 -0
- package/eval/scaffold/python/validation/authoring_rubric.py +326 -0
- package/eval/scaffold/python/validation/comet_workflow.py +187 -0
- package/eval/scaffold/python/validation/core.py +264 -0
- package/eval/scaffold/python/validation/dataset.py +337 -0
- package/eval/scaffold/python/validation/docker.py +106 -0
- package/eval/scaffold/python/validation/evaluator.py +549 -0
- package/eval/scaffold/python/validation/generic_rubric.py +169 -0
- package/eval/scaffold/python/validation/rubric.py +740 -0
- package/eval/scaffold/python/validation/runner.py +237 -0
- package/eval/scaffold/python/validation/scripts.py +58 -0
- package/eval/scaffold/python/validation/tracing.py +313 -0
- package/eval/scaffold/shell/common.sh +126 -0
- package/eval/scaffold/shell/docker.sh +482 -0
- package/eval/scaffold/shell/run-claude-loop.sh +181 -0
- package/eval/scaffold/shell/setup.sh +259 -0
- package/eval/simulator-instruction.md +9 -0
- package/package.json +29 -2
- package/dist/domains/dashboard/web/assets/index-BcO3_QlT.css +0 -1
- package/dist/domains/dashboard/web/assets/index-CjU3VXVu.js +0 -22
|
@@ -10,6 +10,14 @@
|
|
|
10
10
|
|
|
11
11
|
When there is an active comet change (`openspec/changes/<name>/.comet.yaml` exists), **before starting any operation** you must read the `phase` field to confirm the current phase.
|
|
12
12
|
|
|
13
|
+
When multiple active changes exist, resolve the current change first, then run:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
comet state select <change-name>
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Ordinary source writes are governed only by the selected change phase. With multiple active changes and no valid selection, the Hook must block and ask for a choice; it must not guess alphabetically or let an unrelated open, design, or archive change globally block a legal build. A single active change may retain automatic routing.
|
|
20
|
+
|
|
13
21
|
**Phases and allowed operations:**
|
|
14
22
|
|
|
15
23
|
| Phase | Allowed | Prohibited |
|
|
@@ -17,8 +25,8 @@ When there is an active comet change (`openspec/changes/<name>/.comet.yaml` exis
|
|
|
17
25
|
| `open` | Create proposal/design/tasks, run guard | Write source code |
|
|
18
26
|
| `design` | brainstorming, create Design Doc, run guard | Write source code |
|
|
19
27
|
| `build` | Write source code, tests, execute plans | Skip user confirmation points |
|
|
20
|
-
| `verify` | Verification,
|
|
21
|
-
| `archive` | Confirm archive, run archive script | Write source code |
|
|
28
|
+
| `verify` | Verification, record verification report | Skip failure handling, handle the branch early |
|
|
29
|
+
| `archive` | Confirm archive, run archive script, commit archive changes, handle the branch | Write source code |
|
|
22
30
|
|
|
23
31
|
The hook hard interception allowlist includes workflow and platform workspaces such as `openspec/*`, `docs/superpowers/*`, `.superpowers/*`, `.claude/*`, and `.comet/*`; write access to these paths does not allow skipping the current phase's artifacts or confirmation requirements.
|
|
24
32
|
|
|
@@ -36,7 +44,8 @@ Reading the `phase` field alone is not enough — you must also confirm **how**
|
|
|
36
44
|
|
|
37
45
|
Exception: `workflow: hotfix/tweak` intentionally skips design, so an empty `design_doc` is normal and not an illegal jump.
|
|
38
46
|
|
|
39
|
-
Upgrade state note: after a preset (hotfix/tweak) hits an upgrade signal and the user confirms upgrading, `comet
|
|
47
|
+
Upgrade state note: after a preset (hotfix/tweak) hits an upgrade signal and the user confirms upgrading, `comet state transition <name> preset-escalate` legally converts it to `workflow: full` + `phase: design` + `design_doc: null` and clears preset-only build settings. At this point `phase: design` with an empty `design_doc` **is a normal upgrade pre-state**, not an illegal jump — the agent should enter `/comet-design` to supplement the Design Doc, then choose the full workflow configuration again in build. This terminal state does not match the "skipped design" row above (that row only detects `phase: build`).
|
|
48
|
+
|
|
40
49
|
### Skill Invocation (Cannot Replace with Normal Conversation)
|
|
41
50
|
|
|
42
51
|
The following operations must be loaded through the Skill tool. When Skill is unavailable, stop the workflow and prompt to install:
|
|
@@ -51,50 +60,50 @@ The following operations must be loaded through the Skill tool. When Skill is un
|
|
|
51
60
|
|
|
52
61
|
### Script Execution (Cannot Skip)
|
|
53
62
|
|
|
54
|
-
- **Phase exit**: `comet
|
|
55
|
-
- **Compression recovery**: `comet
|
|
56
|
-
- **State update**: After key operations, update fields through `comet
|
|
57
|
-
- **Phase advancement only via guard/transition**: directly running `comet
|
|
58
|
-
- **handoff generation**: `comet
|
|
63
|
+
- **Phase exit**: `comet guard <name> <phase> --apply` (must see ALL CHECKS PASSED)
|
|
64
|
+
- **Compression recovery**: `comet state check <name> <phase> --recover`
|
|
65
|
+
- **State update**: After key operations, update fields through `comet state set`; manually editing .comet.yaml is prohibited
|
|
66
|
+
- **Phase advancement only via guard/transition**: directly running `comet state set <name> phase <value>` to jump phases is prohibited. A preset (hotfix/tweak) upgrade to full must use `comet state transition <name> preset-escalate`
|
|
67
|
+
- **handoff generation**: `comet handoff <name> design --write` (handwriting summaries is prohibited)
|
|
59
68
|
|
|
60
69
|
### User Confirmation (Cannot Auto-Skip)
|
|
61
70
|
|
|
62
71
|
The following decision points must pause to wait for explicit user selection; do not auto-fill based on recommendation rules:
|
|
63
72
|
|
|
64
|
-
- **open**:
|
|
73
|
+
- **open**: Final artifact review, which also confirms the change name and scope. Add an earlier decision only for unresolved target/scope alternatives or a large-PRD split
|
|
65
74
|
- **design**: brainstorming proposal confirmation (Design Doc cannot be created before confirmation)
|
|
66
|
-
- **build**: plan-ready pause
|
|
67
|
-
- **verify**:
|
|
68
|
-
- **archive**: Final confirmation before archiving
|
|
75
|
+
- **build**: After capability preflight, use one joint decision for a plan-ready pause or every executable `isolation` / `build_mode` / `tdd_mode` / `review_mode`; include the branch name when branch is selected, plus large spec-change and preset-upgrade decisions
|
|
76
|
+
- **verify**: Accepting WARNING/SUGGESTION deviations, handling Spec drift, or choosing continue/stop after the automatic repair limit
|
|
77
|
+
- **archive**: Final confirmation before archiving, plus branch-handling selection after the archive commit
|
|
69
78
|
|
|
70
79
|
## Design Phase Specifics
|
|
71
80
|
|
|
72
|
-
1. First script operation = `comet
|
|
81
|
+
1. First script operation = `comet handoff <name> design --write` (loading brainstorming before generating handoff is prohibited)
|
|
73
82
|
2. brainstorming in progress: incrementally update brainstorm-summary.md (update recovery checkpoint after each clarification round or proposal iteration; unconfirmed content marked as pending/candidate)
|
|
74
83
|
3. After brainstorming completes, next step = brainstorm-summary.md finalization → Design Doc → guard
|
|
75
|
-
4.
|
|
84
|
+
4. Active context compaction is optional only after the Design Doc, state evidence, and latest handoff are persisted; when programmatic triggering is unavailable, provide a non-blocking suggestion and continue
|
|
76
85
|
5. **Absolutely cannot start writing implementation code directly** — must first create Design Doc and pass guard
|
|
77
86
|
|
|
78
87
|
## Build Phase Specifics
|
|
79
88
|
|
|
80
|
-
1. After plan creation,
|
|
89
|
+
1. After plan creation, filter unavailable options, then ask one joint question: pause, or submit workspace/execution/TDD/review settings and any conditional branch name together
|
|
81
90
|
2. After each task acceptance, must: tasks.md checkmark → git commit (do not accumulate). `subagent-driven-development` must complete acceptance according to the current `review_mode`, then the coordinator performs targeted verification by unique task text; do not use an incomplete task summary table to replace current task verification
|
|
82
91
|
3. When encountering failures, must load **systematic-debugging** skill; do not propose source code fixes before root cause is located
|
|
83
92
|
4. spec change grading: small changes edit directly | medium changes load brainstorming | large changes pause and wait for user confirmation to split
|
|
84
93
|
|
|
85
94
|
## Verify Phase Specifics
|
|
86
95
|
|
|
87
|
-
1. First step run `comet
|
|
88
|
-
2.
|
|
89
|
-
3. After 3
|
|
90
|
-
4.
|
|
96
|
+
1. First step run `comet state scale <name>` to determine verification level
|
|
97
|
+
2. For the first 3 clearly repairable failures, automatically run `comet state transition <name> verify-fail`, return to build, and enter `/comet-build`. CRITICAL/IMPORTANT failures cannot be accepted as deviations
|
|
98
|
+
3. The state machine owns `verify_failures`. After it reaches `3`, the next failure must ask whether to continue fixing or stop the workflow for an external decision
|
|
99
|
+
4. Ask whether to fix or accept WARNING/SUGGESTION only when repair introduces a behavior, scope, or risk tradeoff. Safe, local, tradeoff-free repairs close automatically
|
|
91
100
|
|
|
92
101
|
## Context Compression Recovery
|
|
93
102
|
|
|
94
103
|
If context compression is suspected (previous conversation was summarized, previous discussion cannot be found), immediately run:
|
|
95
104
|
|
|
96
105
|
```bash
|
|
97
|
-
|
|
106
|
+
comet state check <name> <phase> --recover
|
|
98
107
|
```
|
|
99
108
|
|
|
100
109
|
Decide next step according to the script's **Recovery action** output.
|
|
@@ -115,17 +124,11 @@ After recovery, first re-run the "Phase-Entry Self-Consistency Check" table: if
|
|
|
115
124
|
After guard `--apply` succeeds, do not hardcode the next skill in this rule. First run:
|
|
116
125
|
|
|
117
126
|
```bash
|
|
118
|
-
comet
|
|
119
|
-
```
|
|
120
|
-
|
|
121
|
-
If `comet-env.mjs` has already located the scripts, the equivalent command is:
|
|
122
|
-
|
|
123
|
-
```bash
|
|
124
|
-
node "$COMET_STATE" next <change-name>
|
|
127
|
+
comet state next <change-name>
|
|
125
128
|
```
|
|
126
129
|
|
|
127
130
|
Decide the next step from the script output:
|
|
128
131
|
|
|
129
132
|
- `NEXT: auto` → use the Skill tool to load the skill named by `SKILL`
|
|
130
|
-
- `NEXT: manual` → do not load the next skill;
|
|
133
|
+
- `NEXT: manual` → do not load the next skill; return control with `HINT`, end the invocation, and do not create another confirmation point
|
|
131
134
|
- `NEXT: done` → the workflow is complete; no further action is needed
|
|
@@ -10,6 +10,14 @@
|
|
|
10
10
|
|
|
11
11
|
有活跃 comet change 时(`openspec/changes/<name>/.comet.yaml` 存在),**每次开始执行操作前**必须读取 `phase` 字段确认当前阶段。
|
|
12
12
|
|
|
13
|
+
当存在多个 active change 时,必须先明确当前 change,再运行:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
comet state select <change-name>
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
普通源码写入只受已选择 change 的阶段约束。多个 active change 且没有有效选择时,Hook 必须阻塞并提示选择;不得按字母序猜测,也不得让无关 change 的 `open`、`design` 或 `archive` 阶段全局封锁合法 build。单 active change 无选择时可保持自动归属。
|
|
20
|
+
|
|
13
21
|
**阶段与允许操作:**
|
|
14
22
|
|
|
15
23
|
| 阶段 | 允许 | 禁止 |
|
|
@@ -17,8 +25,8 @@
|
|
|
17
25
|
| `open` | 创建 proposal/design/tasks, 运行 guard | 写源代码 |
|
|
18
26
|
| `design` | brainstorming, 创建 Design Doc, 运行 guard | 写源代码 |
|
|
19
27
|
| `build` | 写源代码、测试、执行计划 | 跳过用户确认点 |
|
|
20
|
-
| `verify` |
|
|
21
|
-
| `archive` |
|
|
28
|
+
| `verify` | 验证、记录验证报告 | 跳过失败处理、提前处理分支 |
|
|
29
|
+
| `archive` | 确认归档、运行归档脚本、提交归档改动、分支处理 | 写源代码 |
|
|
22
30
|
|
|
23
31
|
Hook 硬拦截白名单包括 `openspec/*`、`docs/superpowers/*`、`.superpowers/*`、`.claude/*` 和 `.comet/*` 等流程/平台工作区;这些路径可写不代表可以跳过当前阶段的产物和确认要求。
|
|
24
32
|
|
|
@@ -32,11 +40,11 @@ Hook 硬拦截白名单包括 `openspec/*`、`docs/superpowers/*`、`.superpower
|
|
|
32
40
|
| `phase: build` + proposal/design/tasks 任一缺失或为空 | 绕过 open 空跳 | 回 `/comet-open` 补齐三件套 |
|
|
33
41
|
| `phase: archive` + `verify_result` ≠ `pass` | 绕过 verify 空跳 | 回 `/comet-verify` 完成验证 |
|
|
34
42
|
|
|
35
|
-
> 说明:上表只覆盖 hook
|
|
43
|
+
> 说明:上表只覆盖 hook 硬拦截实际检测的范围(`design_doc` 空跳在 build 阶段;proposal/design/tasks 三件套完整性在 open→build 的 guard 退出时校验)。`verify` 阶段不在此写源码自洽检查内——若 verify 发现产物缺失,按下方「Verify 阶段专项」的 verify-fail 回退处理。
|
|
36
44
|
|
|
37
45
|
预设例外:`workflow: hotfix/tweak` 本就跳过 design,`design_doc` 为空属正常,不算非法。
|
|
38
46
|
|
|
39
|
-
升级态说明:预设(hotfix/tweak)命中升级信号并经用户确认升级后,通过 `comet
|
|
47
|
+
升级态说明:预设(hotfix/tweak)命中升级信号并经用户确认升级后,通过 `comet state transition <name> preset-escalate` 合法地变为 `workflow: full` + `phase: design` + `design_doc: null`,同时清除预设专属 build 配置。此时 `phase: design` + `design_doc` 为空**属正常升级前置态**,不是非法空跳——agent 应进入 `/comet-design` 补 Design Doc,并在 build 重新完成联合工作方式选择。该终态不命中上表「绕过 design 空跳」行(该行仅检测 `phase: build`)。
|
|
40
48
|
|
|
41
49
|
### Skill 调用(不可用普通对话替代)
|
|
42
50
|
|
|
@@ -52,50 +60,50 @@ Hook 硬拦截白名单包括 `openspec/*`、`docs/superpowers/*`、`.superpower
|
|
|
52
60
|
|
|
53
61
|
### 脚本执行(不可跳过)
|
|
54
62
|
|
|
55
|
-
- **阶段退出**: `comet
|
|
56
|
-
- **压缩恢复**: `comet
|
|
57
|
-
- **状态更新**: 关键操作后通过 `comet
|
|
58
|
-
- **阶段推进只能经 guard/transition**: 禁止用 `comet
|
|
59
|
-
- **handoff 生成**: `comet
|
|
63
|
+
- **阶段退出**: `comet guard <name> <phase> --apply`(必须看到 ALL CHECKS PASSED)
|
|
64
|
+
- **压缩恢复**: `comet state check <name> <phase> --recover`
|
|
65
|
+
- **状态更新**: 关键操作后通过 `comet state set` 更新字段,禁止手工编辑 .comet.yaml
|
|
66
|
+
- **阶段推进只能经 guard/transition**: 禁止用 `comet state set <name> phase <值>` 手动跳阶段;预设升级到 full 必须用 `comet state transition <name> preset-escalate`
|
|
67
|
+
- **handoff 生成**: `comet handoff <name> design --write`(禁止手写摘要)
|
|
60
68
|
|
|
61
69
|
### 用户确认(不可自动跳过)
|
|
62
70
|
|
|
63
71
|
以下决策点必须暂停等待用户明确选择,不得根据推荐规则自动填写:
|
|
64
72
|
|
|
65
|
-
- **open**:
|
|
73
|
+
- **open**: artifact 最终审视(同时确认 change 名称与范围);只有目标/范围仍有互斥选择或大型 PRD 拆分时增加前置决策
|
|
66
74
|
- **design**: brainstorming 方案确认(确认前不得创建 Design Doc)
|
|
67
|
-
- **build**: plan-ready
|
|
68
|
-
- **verify**:
|
|
69
|
-
- **archive**:
|
|
75
|
+
- **build**: 能力预检后,在一个联合决策中选择 plan-ready 暂停或所有可执行的 `isolation` / `build_mode` / `tdd_mode` / `review_mode`;选择 branch 时同时确认分支名,另含 spec 大规模变更确认、预设升级判定
|
|
76
|
+
- **verify**: 接受 WARNING/SUGGESTION 偏差、Spec 漂移处理,或超过自动修复上限后的继续/停止策略
|
|
77
|
+
- **archive**: 归档前最终确认,以及归档提交后的 branch handling 选择
|
|
70
78
|
|
|
71
79
|
## Design 阶段专项
|
|
72
80
|
|
|
73
|
-
1. 第一个脚本操作 = `comet
|
|
81
|
+
1. 第一个脚本操作 = `comet handoff <name> design --write`(未生成 handoff 禁止加载 brainstorming)
|
|
74
82
|
2. brainstorming in progress: incrementally update brainstorm-summary.md(每轮澄清或方案迭代后增量更新恢复检查点,未确认内容标注为待确认/候选)
|
|
75
83
|
3. brainstorming 完成后下一步 = brainstorm-summary.md 定稿 → Design Doc → guard
|
|
76
|
-
4.
|
|
84
|
+
4. 主动式上下文压缩只能在 Design Doc、状态和最新 handoff 落盘后按需执行;无法程序化触发时给出非阻塞建议并继续
|
|
77
85
|
5. **绝对不能直接开始写实现代码** — 必须先创建 Design Doc 并通过 guard
|
|
78
86
|
|
|
79
87
|
## Build 阶段专项
|
|
80
88
|
|
|
81
|
-
1. plan
|
|
89
|
+
1. plan 创建后先过滤不可执行选项,再只发起一个联合决策:暂停,或一次性提交工作区/执行/TDD/审查配置及条件性的分支名
|
|
82
90
|
2. 每个 task 验收后必须: tasks.md 打勾 → git commit(不得积攒)。`subagent-driven-development` 必须按当前 `review_mode` 完成验收,再由协调者按任务唯一文本定向勾选和验证;不得用未完成任务总表代替当前任务验证
|
|
83
91
|
3. 遇到失败必须加载 **systematic-debugging** skill,根因未定位前不得提出源码修复
|
|
84
92
|
4. spec 变更分级: 小改直接编辑 | 中改加载 brainstorming | 大改暂停等用户确认拆分
|
|
85
93
|
|
|
86
94
|
## Verify 阶段专项
|
|
87
95
|
|
|
88
|
-
1. 第一步运行 `comet
|
|
89
|
-
2.
|
|
90
|
-
3.
|
|
91
|
-
4.
|
|
96
|
+
1. 第一步运行 `comet state scale <name>` 确定验证级别
|
|
97
|
+
2. 前 3 次明确可修复失败自动运行 `comet state transition <name> verify-fail` 回到 build,并进入 `/comet-build`;CRITICAL/IMPORTANT 不得接受偏差
|
|
98
|
+
3. `verify_failures` 由状态机维护。达到 `3` 后的下一次失败必须让用户选择继续修复或停止 workflow 寻求外部决策
|
|
99
|
+
4. WARNING/SUGGESTION 只有在修复涉及行为、范围或风险取舍时才询问修复/接受;安全、局部、无取舍的修复自动闭环
|
|
92
100
|
|
|
93
101
|
## 上下文压缩恢复
|
|
94
102
|
|
|
95
103
|
如果怀疑发生上下文压缩(之前对话被摘要、找不到之前讨论的内容),立即运行:
|
|
96
104
|
|
|
97
105
|
```bash
|
|
98
|
-
|
|
106
|
+
comet state check <name> <phase> --recover
|
|
99
107
|
```
|
|
100
108
|
|
|
101
109
|
按脚本输出的 **Recovery action** 决定下一步。
|
|
@@ -116,17 +124,11 @@ node "$COMET_STATE" check <name> <phase> --recover
|
|
|
116
124
|
guard `--apply` 成功后,不得在本规则中硬编码下一阶段 skill。必须先运行:
|
|
117
125
|
|
|
118
126
|
```bash
|
|
119
|
-
comet
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
若已通过 `comet-env.mjs` 定位脚本,等价运行:
|
|
123
|
-
|
|
124
|
-
```bash
|
|
125
|
-
node "$COMET_STATE" next <change-name>
|
|
127
|
+
comet state next <change-name>
|
|
126
128
|
```
|
|
127
129
|
|
|
128
130
|
按脚本输出决定下一步:
|
|
129
131
|
|
|
130
132
|
- `NEXT: auto` → 使用 Skill 工具加载 `SKILL` 指向的 skill
|
|
131
|
-
- `NEXT: manual` → 不加载下一 skill,按 `HINT`
|
|
133
|
+
- `NEXT: manual` → 不加载下一 skill,按 `HINT` 交还控制权并结束当前调用;不再创建确认点
|
|
132
134
|
- `NEXT: done` → 流程已完成,无需继续
|