@rpamis/comet 0.4.0-beta.3 → 0.4.0-beta.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (605) hide show
  1. package/LICENSE +21 -21
  2. package/README.md +15 -1
  3. package/assets/manifest.json +2 -1
  4. package/assets/skills/comet/SKILL.md +54 -28
  5. package/assets/skills/comet/reference/auto-transition.md +1 -1
  6. package/assets/skills/comet/reference/comet-yaml-fields.md +11 -6
  7. package/assets/skills/comet/reference/context-recovery.md +12 -2
  8. package/assets/skills/comet/reference/decision-point.md +11 -0
  9. package/assets/skills/comet/reference/scripts.md +47 -15
  10. package/assets/skills/comet/reference/subagent-dispatch.md +6 -5
  11. package/assets/skills/comet/rules/comet-phase-guard.en.md +31 -28
  12. package/assets/skills/comet/rules/comet-phase-guard.md +31 -29
  13. package/assets/skills/comet/scripts/comet-resume-probe.mjs +3 -0
  14. package/assets/skills/comet/scripts/comet-runtime.mjs +1568 -476
  15. package/assets/skills/comet-any/SKILL.md +1 -1
  16. package/assets/skills/comet-any/reference/authored-zone-example.md +10 -5
  17. package/assets/skills/comet-any/reference/subagents/pause-points-author.md +14 -11
  18. package/assets/skills/comet-any/reference/subagents/skill-reviewer.md +2 -0
  19. package/assets/skills/comet-any/reference/subagents/workflow-entry-author.md +5 -4
  20. package/assets/skills/comet-archive/SKILL.md +52 -18
  21. package/assets/skills/comet-build/SKILL.md +64 -45
  22. package/assets/skills/comet-design/SKILL.md +24 -22
  23. package/assets/skills/comet-hotfix/SKILL.md +37 -34
  24. package/assets/skills/comet-open/SKILL.md +106 -57
  25. package/assets/skills/comet-tweak/SKILL.md +23 -18
  26. package/assets/skills/comet-verify/SKILL.md +57 -72
  27. package/assets/skills-zh/comet/SKILL.md +54 -28
  28. package/assets/skills-zh/comet/reference/auto-transition.md +1 -1
  29. package/assets/skills-zh/comet/reference/comet-yaml-fields.md +11 -6
  30. package/assets/skills-zh/comet/reference/context-recovery.md +12 -2
  31. package/assets/skills-zh/comet/reference/decision-point.md +11 -0
  32. package/assets/skills-zh/comet/reference/scripts.md +47 -15
  33. package/assets/skills-zh/comet/reference/subagent-dispatch.md +6 -5
  34. package/assets/skills-zh/comet-any/SKILL.md +1 -1
  35. package/assets/skills-zh/comet-any/reference/authored-zone-example.md +10 -5
  36. package/assets/skills-zh/comet-any/reference/subagents/pause-points-author.md +14 -12
  37. package/assets/skills-zh/comet-any/reference/subagents/skill-reviewer.md +2 -0
  38. package/assets/skills-zh/comet-any/reference/subagents/workflow-entry-author.md +5 -4
  39. package/assets/skills-zh/comet-archive/SKILL.md +52 -18
  40. package/assets/skills-zh/comet-build/SKILL.md +64 -45
  41. package/assets/skills-zh/comet-design/SKILL.md +24 -22
  42. package/assets/skills-zh/comet-hotfix/SKILL.md +36 -33
  43. package/assets/skills-zh/comet-open/SKILL.md +105 -56
  44. package/assets/skills-zh/comet-tweak/SKILL.md +22 -17
  45. package/assets/skills-zh/comet-verify/SKILL.md +57 -72
  46. package/dist/app/cli/comet-banner.d.ts +25 -0
  47. package/dist/app/cli/comet-banner.d.ts.map +1 -0
  48. package/dist/app/cli/comet-banner.js +265 -0
  49. package/dist/app/cli/comet-banner.js.map +1 -0
  50. package/dist/app/cli/index.js +36 -2
  51. package/dist/app/cli/index.js.map +1 -1
  52. package/dist/app/commands/classic.d.ts +4 -0
  53. package/dist/app/commands/classic.d.ts.map +1 -0
  54. package/dist/app/commands/classic.js +11 -0
  55. package/dist/app/commands/classic.js.map +1 -0
  56. package/dist/app/commands/doctor.d.ts.map +1 -1
  57. package/dist/app/commands/doctor.js +99 -37
  58. package/dist/app/commands/doctor.js.map +1 -1
  59. package/dist/app/commands/eval.d.ts.map +1 -1
  60. package/dist/app/commands/eval.js +57 -20
  61. package/dist/app/commands/eval.js.map +1 -1
  62. package/dist/app/commands/i18n.d.ts +1 -1
  63. package/dist/app/commands/i18n.d.ts.map +1 -1
  64. package/dist/app/commands/i18n.js +12 -4
  65. package/dist/app/commands/i18n.js.map +1 -1
  66. package/dist/app/commands/init.d.ts +1 -1
  67. package/dist/app/commands/init.d.ts.map +1 -1
  68. package/dist/app/commands/init.js +72 -25
  69. package/dist/app/commands/init.js.map +1 -1
  70. package/dist/app/commands/project-scope-selection.d.ts +13 -0
  71. package/dist/app/commands/project-scope-selection.d.ts.map +1 -0
  72. package/dist/app/commands/project-scope-selection.js +33 -0
  73. package/dist/app/commands/project-scope-selection.js.map +1 -0
  74. package/dist/app/commands/resume-probe.d.ts +11 -0
  75. package/dist/app/commands/resume-probe.d.ts.map +1 -0
  76. package/dist/app/commands/resume-probe.js +63 -0
  77. package/dist/app/commands/resume-probe.js.map +1 -0
  78. package/dist/app/commands/status.d.ts +31 -0
  79. package/dist/app/commands/status.d.ts.map +1 -1
  80. package/dist/app/commands/status.js +109 -20
  81. package/dist/app/commands/status.js.map +1 -1
  82. package/dist/app/commands/uninstall.d.ts +5 -0
  83. package/dist/app/commands/uninstall.d.ts.map +1 -1
  84. package/dist/app/commands/uninstall.js +264 -53
  85. package/dist/app/commands/uninstall.js.map +1 -1
  86. package/dist/app/commands/update.d.ts +6 -0
  87. package/dist/app/commands/update.d.ts.map +1 -1
  88. package/dist/app/commands/update.js +558 -86
  89. package/dist/app/commands/update.js.map +1 -1
  90. package/dist/config/repository-layout.json +4 -2
  91. package/dist/domains/bundle/bundle-platform.d.ts +1 -0
  92. package/dist/domains/bundle/bundle-platform.d.ts.map +1 -1
  93. package/dist/domains/bundle/bundle-platform.js +9 -3
  94. package/dist/domains/bundle/bundle-platform.js.map +1 -1
  95. package/dist/domains/bundle/eval-manifest-runtime.d.ts +6 -0
  96. package/dist/domains/bundle/eval-manifest-runtime.d.ts.map +1 -0
  97. package/dist/domains/bundle/eval-manifest-runtime.js +58 -0
  98. package/dist/domains/bundle/eval-manifest-runtime.js.map +1 -0
  99. package/dist/domains/bundle/types.d.ts +1 -1
  100. package/dist/domains/bundle/types.d.ts.map +1 -1
  101. package/dist/domains/comet-classic/classic-archive.d.ts +1 -0
  102. package/dist/domains/comet-classic/classic-archive.d.ts.map +1 -1
  103. package/dist/domains/comet-classic/classic-archive.js +35 -38
  104. package/dist/domains/comet-classic/classic-archive.js.map +1 -1
  105. package/dist/domains/comet-classic/classic-cli.d.ts +1 -1
  106. package/dist/domains/comet-classic/classic-cli.d.ts.map +1 -1
  107. package/dist/domains/comet-classic/classic-cli.js +3 -0
  108. package/dist/domains/comet-classic/classic-cli.js.map +1 -1
  109. package/dist/domains/comet-classic/classic-command-checks.d.ts +20 -0
  110. package/dist/domains/comet-classic/classic-command-checks.d.ts.map +1 -0
  111. package/dist/domains/comet-classic/classic-command-checks.js +97 -0
  112. package/dist/domains/comet-classic/classic-command-checks.js.map +1 -0
  113. package/dist/domains/comet-classic/classic-current-change.d.ts +19 -0
  114. package/dist/domains/comet-classic/classic-current-change.d.ts.map +1 -0
  115. package/dist/domains/comet-classic/classic-current-change.js +132 -0
  116. package/dist/domains/comet-classic/classic-current-change.js.map +1 -0
  117. package/dist/domains/comet-classic/classic-diagnostics.d.ts +2 -1
  118. package/dist/domains/comet-classic/classic-diagnostics.d.ts.map +1 -1
  119. package/dist/domains/comet-classic/classic-diagnostics.js +54 -1
  120. package/dist/domains/comet-classic/classic-diagnostics.js.map +1 -1
  121. package/dist/domains/comet-classic/classic-evidence.d.ts.map +1 -1
  122. package/dist/domains/comet-classic/classic-evidence.js +4 -0
  123. package/dist/domains/comet-classic/classic-evidence.js.map +1 -1
  124. package/dist/domains/comet-classic/classic-guard.d.ts.map +1 -1
  125. package/dist/domains/comet-classic/classic-guard.js +83 -80
  126. package/dist/domains/comet-classic/classic-guard.js.map +1 -1
  127. package/dist/domains/comet-classic/classic-hook-guard.d.ts.map +1 -1
  128. package/dist/domains/comet-classic/classic-hook-guard.js +179 -42
  129. package/dist/domains/comet-classic/classic-hook-guard.js.map +1 -1
  130. package/dist/domains/comet-classic/classic-project-config.d.ts +16 -0
  131. package/dist/domains/comet-classic/classic-project-config.d.ts.map +1 -0
  132. package/dist/domains/comet-classic/classic-project-config.js +40 -0
  133. package/dist/domains/comet-classic/classic-project-config.js.map +1 -0
  134. package/dist/domains/comet-classic/classic-resolver.d.ts.map +1 -1
  135. package/dist/domains/comet-classic/classic-resolver.js +4 -4
  136. package/dist/domains/comet-classic/classic-resolver.js.map +1 -1
  137. package/dist/domains/comet-classic/classic-resume-probe-command.d.ts +3 -0
  138. package/dist/domains/comet-classic/classic-resume-probe-command.d.ts.map +1 -0
  139. package/dist/domains/comet-classic/classic-resume-probe-command.js +58 -0
  140. package/dist/domains/comet-classic/classic-resume-probe-command.js.map +1 -0
  141. package/dist/domains/comet-classic/classic-resume-probe-entry.d.ts +2 -0
  142. package/dist/domains/comet-classic/classic-resume-probe-entry.d.ts.map +1 -0
  143. package/dist/domains/comet-classic/classic-resume-probe-entry.js +4 -0
  144. package/dist/domains/comet-classic/classic-resume-probe-entry.js.map +1 -0
  145. package/dist/domains/comet-classic/classic-resume-probe.d.ts +29 -0
  146. package/dist/domains/comet-classic/classic-resume-probe.d.ts.map +1 -0
  147. package/dist/domains/comet-classic/classic-resume-probe.js +386 -0
  148. package/dist/domains/comet-classic/classic-resume-probe.js.map +1 -0
  149. package/dist/domains/comet-classic/classic-runtime-run.d.ts +2 -1
  150. package/dist/domains/comet-classic/classic-runtime-run.d.ts.map +1 -1
  151. package/dist/domains/comet-classic/classic-runtime-run.js +38 -0
  152. package/dist/domains/comet-classic/classic-runtime-run.js.map +1 -1
  153. package/dist/domains/comet-classic/classic-state-command.d.ts.map +1 -1
  154. package/dist/domains/comet-classic/classic-state-command.js +150 -25
  155. package/dist/domains/comet-classic/classic-state-command.js.map +1 -1
  156. package/dist/domains/comet-classic/classic-state.d.ts +5 -2
  157. package/dist/domains/comet-classic/classic-state.d.ts.map +1 -1
  158. package/dist/domains/comet-classic/classic-state.js +17 -1
  159. package/dist/domains/comet-classic/classic-state.js.map +1 -1
  160. package/dist/domains/comet-classic/classic-store.d.ts +4 -1
  161. package/dist/domains/comet-classic/classic-store.d.ts.map +1 -1
  162. package/dist/domains/comet-classic/classic-store.js +4 -3
  163. package/dist/domains/comet-classic/classic-store.js.map +1 -1
  164. package/dist/domains/comet-classic/classic-transitions.d.ts +1 -1
  165. package/dist/domains/comet-classic/classic-transitions.d.ts.map +1 -1
  166. package/dist/domains/comet-classic/classic-transitions.js +36 -3
  167. package/dist/domains/comet-classic/classic-transitions.js.map +1 -1
  168. package/dist/domains/comet-classic/classic-validate-command.d.ts.map +1 -1
  169. package/dist/domains/comet-classic/classic-validate-command.js +8 -1
  170. package/dist/domains/comet-classic/classic-validate-command.js.map +1 -1
  171. package/dist/domains/comet-classic/index.d.ts +3 -0
  172. package/dist/domains/comet-classic/index.d.ts.map +1 -1
  173. package/dist/domains/comet-classic/index.js +3 -0
  174. package/dist/domains/comet-classic/index.js.map +1 -1
  175. package/dist/domains/dashboard/collector.js +10 -2
  176. package/dist/domains/dashboard/collector.js.map +1 -1
  177. package/dist/domains/dashboard/web/assets/_commonjsHelpers-CqkleIqs.js +1 -0
  178. package/dist/domains/dashboard/web/assets/arc-TGVUG2WN.js +1 -0
  179. package/dist/domains/dashboard/web/assets/architectureDiagram-3BPJPVTR-ASHmiHoP.js +36 -0
  180. package/dist/domains/dashboard/web/assets/blockDiagram-GPEHLZMM-CwASxBKe.js +132 -0
  181. package/dist/domains/dashboard/web/assets/c4Diagram-AAUBKEIU-BU8z_WNk.js +10 -0
  182. package/dist/domains/dashboard/web/assets/channel-kp6Ng2JJ.js +1 -0
  183. package/dist/domains/dashboard/web/assets/chunk-2J33WTMH-CBGlUx4_.js +1 -0
  184. package/dist/domains/dashboard/web/assets/chunk-4BX2VUAB-CS-iGcFz.js +1 -0
  185. package/dist/domains/dashboard/web/assets/chunk-55IACEB6-BUtCVJat.js +1 -0
  186. package/dist/domains/dashboard/web/assets/chunk-727SXJPM-C94tj_pf.js +206 -0
  187. package/dist/domains/dashboard/web/assets/chunk-AQP2D5EJ-BFOKcpdG.js +231 -0
  188. package/dist/domains/dashboard/web/assets/chunk-FMBD7UC4-Dmx8B-st.js +15 -0
  189. package/dist/domains/dashboard/web/assets/chunk-ND2GUHAM-DYV00CNn.js +1 -0
  190. package/dist/domains/dashboard/web/assets/chunk-QZHKN3VN-DMJJyQnw.js +1 -0
  191. package/dist/domains/dashboard/web/assets/classDiagram-4FO5ZUOK-yBDjgMY_.js +1 -0
  192. package/dist/domains/dashboard/web/assets/classDiagram-v2-Q7XG4LA2-yBDjgMY_.js +1 -0
  193. package/dist/domains/dashboard/web/assets/cose-bilkent-S5V4N54A-P4nE2rea.js +1 -0
  194. package/dist/domains/dashboard/web/assets/cynefin-VYW2F7L2-C-R1HXDv.js +166 -0
  195. package/dist/domains/dashboard/web/assets/cytoscape.esm-D3_iZ_3b.js +321 -0
  196. package/dist/domains/dashboard/web/assets/dagre-BM42HDAG-BH8d-C9z.js +4 -0
  197. package/dist/domains/dashboard/web/assets/defaultLocale-DX6XiGOO.js +1 -0
  198. package/dist/domains/dashboard/web/assets/diagram-2AECGRRQ-DpXTgQIH.js +43 -0
  199. package/dist/domains/dashboard/web/assets/diagram-5GNKFQAL-DSiFJzSw.js +10 -0
  200. package/dist/domains/dashboard/web/assets/diagram-KO2AKTUF-DULMkben.js +3 -0
  201. package/dist/domains/dashboard/web/assets/diagram-LMA3HP47-Pi8m_NO9.js +24 -0
  202. package/dist/domains/dashboard/web/assets/diagram-OG6HWLK6-BSe6bCWm.js +24 -0
  203. package/dist/domains/dashboard/web/assets/erDiagram-TEJ5UH35-zb-FEcbq.js +85 -0
  204. package/dist/domains/dashboard/web/assets/flowDiagram-I6XJVG4X-Dr-Gk0z2.js +162 -0
  205. package/dist/domains/dashboard/web/assets/ganttDiagram-6RSMTGT7-qs6Zb2A6.js +292 -0
  206. package/dist/domains/dashboard/web/assets/gitGraphDiagram-PVQCEYII-BQ314orT.js +106 -0
  207. package/dist/domains/dashboard/web/assets/graph--OzhPTMs.js +1 -0
  208. package/dist/domains/dashboard/web/assets/index-BLAfkNTu.css +1 -0
  209. package/dist/domains/dashboard/web/assets/index-DlSYK56b.js +139 -0
  210. package/dist/domains/dashboard/web/assets/index-DtLUOcAW.js +24 -0
  211. package/dist/domains/dashboard/web/assets/index-QO0z6F4w.js +10 -0
  212. package/dist/domains/dashboard/web/assets/infoDiagram-5YYISTIA-O0bQp98A.js +2 -0
  213. package/dist/domains/dashboard/web/assets/init-Gi6I4Gst.js +1 -0
  214. package/dist/domains/dashboard/web/assets/ishikawaDiagram-YF4QCWOH-BPC7wbKh.js +70 -0
  215. package/dist/domains/dashboard/web/assets/journeyDiagram-JHISSGLW-ohViOd4b.js +139 -0
  216. package/dist/domains/dashboard/web/assets/kanban-definition-UN3LZRKU-CHleNdbG.js +89 -0
  217. package/dist/domains/dashboard/web/assets/katex-HP8lGamR.js +257 -0
  218. package/dist/domains/dashboard/web/assets/layout-SsrduOYp.js +1 -0
  219. package/dist/domains/dashboard/web/assets/linear-CyqiW--Q.js +1 -0
  220. package/dist/domains/dashboard/web/assets/marked.esm-DECB3bxB.js +64 -0
  221. package/dist/domains/dashboard/web/assets/mermaid.core-qS9XvtaK.js +301 -0
  222. package/dist/domains/dashboard/web/assets/mindmap-definition-RKZ34NQL-oaGkqO9l.js +96 -0
  223. package/dist/domains/dashboard/web/assets/ordinal-Cboi1Yqb.js +1 -0
  224. package/dist/domains/dashboard/web/assets/pieDiagram-4H26LBE5-CPcB-AFh.js +30 -0
  225. package/dist/domains/dashboard/web/assets/purify.es-VaSPOPhr.js +3 -0
  226. package/dist/domains/dashboard/web/assets/quadrantDiagram-W4KKPZXB-LFy52Qtj.js +7 -0
  227. package/dist/domains/dashboard/web/assets/requirementDiagram-4Y6WPE33-BdjeaYP_.js +84 -0
  228. package/dist/domains/dashboard/web/assets/sankeyDiagram-5OEKKPKP-DDJ4styL.js +40 -0
  229. package/dist/domains/dashboard/web/assets/sequenceDiagram-3UESZ5HK-BL5bFNoz.js +162 -0
  230. package/dist/domains/dashboard/web/assets/stateDiagram-AJRCARHV-BNUfikui.js +1 -0
  231. package/dist/domains/dashboard/web/assets/stateDiagram-v2-BHNVJYJU-DGrsmWts.js +1 -0
  232. package/dist/domains/dashboard/web/assets/timeline-definition-PNZ67QCA--ps2uu6L.js +120 -0
  233. package/dist/domains/dashboard/web/assets/vennDiagram-CIIHVFJN-C2q8q26c.js +34 -0
  234. package/dist/domains/dashboard/web/assets/wardleyDiagram-YWT4CUSO-BUH9gXqd.js +78 -0
  235. package/dist/domains/dashboard/web/assets/xychartDiagram-2RQKCTM6-B-DielEy.js +7 -0
  236. package/dist/domains/dashboard/web/index.html +16 -16
  237. package/dist/domains/engine/types.d.ts +1 -1
  238. package/dist/domains/engine/types.d.ts.map +1 -1
  239. package/dist/domains/factory/package.d.ts.map +1 -1
  240. package/dist/domains/factory/package.js +20 -6
  241. package/dist/domains/factory/package.js.map +1 -1
  242. package/dist/domains/integrations/openspec.d.ts +4 -1
  243. package/dist/domains/integrations/openspec.d.ts.map +1 -1
  244. package/dist/domains/integrations/openspec.js +62 -10
  245. package/dist/domains/integrations/openspec.js.map +1 -1
  246. package/dist/domains/skill/find.d.ts.map +1 -1
  247. package/dist/domains/skill/find.js +1 -6
  248. package/dist/domains/skill/find.js.map +1 -1
  249. package/dist/domains/skill/json-object.d.ts +13 -0
  250. package/dist/domains/skill/json-object.d.ts.map +1 -0
  251. package/dist/domains/skill/json-object.js +24 -0
  252. package/dist/domains/skill/json-object.js.map +1 -0
  253. package/dist/domains/skill/managed-markdown.d.ts +14 -0
  254. package/dist/domains/skill/managed-markdown.d.ts.map +1 -0
  255. package/dist/domains/skill/managed-markdown.js +126 -0
  256. package/dist/domains/skill/managed-markdown.js.map +1 -0
  257. package/dist/domains/skill/platform-inspect.d.ts +9 -0
  258. package/dist/domains/skill/platform-inspect.d.ts.map +1 -0
  259. package/dist/domains/skill/platform-inspect.js +149 -0
  260. package/dist/domains/skill/platform-inspect.js.map +1 -0
  261. package/dist/domains/skill/platform-install.d.ts +20 -9
  262. package/dist/domains/skill/platform-install.d.ts.map +1 -1
  263. package/dist/domains/skill/platform-install.js +356 -153
  264. package/dist/domains/skill/platform-install.js.map +1 -1
  265. package/dist/domains/skill/project-instructions.d.ts +22 -0
  266. package/dist/domains/skill/project-instructions.d.ts.map +1 -0
  267. package/dist/domains/skill/project-instructions.js +61 -0
  268. package/dist/domains/skill/project-instructions.js.map +1 -0
  269. package/dist/domains/skill/uninstall.d.ts +3 -1
  270. package/dist/domains/skill/uninstall.d.ts.map +1 -1
  271. package/dist/domains/skill/uninstall.js +259 -149
  272. package/dist/domains/skill/uninstall.js.map +1 -1
  273. package/dist/platform/fs/file-system.d.ts +3 -3
  274. package/dist/platform/fs/file-system.d.ts.map +1 -1
  275. package/dist/platform/fs/file-system.js +21 -11
  276. package/dist/platform/fs/file-system.js.map +1 -1
  277. package/dist/platform/install/detect.d.ts +2 -1
  278. package/dist/platform/install/detect.d.ts.map +1 -1
  279. package/dist/platform/install/detect.js +10 -1
  280. package/dist/platform/install/detect.js.map +1 -1
  281. package/dist/platform/install/platforms.d.ts +9 -1
  282. package/dist/platform/install/platforms.d.ts.map +1 -1
  283. package/dist/platform/install/platforms.js +23 -4
  284. package/dist/platform/install/platforms.js.map +1 -1
  285. package/dist/platform/install/project-registry.d.ts +38 -0
  286. package/dist/platform/install/project-registry.d.ts.map +1 -0
  287. package/dist/platform/install/project-registry.js +213 -0
  288. package/dist/platform/install/project-registry.js.map +1 -0
  289. package/dist/platform/install/skill-root-owner.d.ts +15 -0
  290. package/dist/platform/install/skill-root-owner.d.ts.map +1 -0
  291. package/dist/platform/install/skill-root-owner.js +68 -0
  292. package/dist/platform/install/skill-root-owner.js.map +1 -0
  293. package/eval/.env.example +52 -0
  294. package/eval/CLAUDE.md +43 -0
  295. package/eval/README.md +538 -0
  296. package/eval/langsmith/.env.example +16 -0
  297. package/eval/langsmith/README.md +93 -0
  298. package/eval/langsmith/skills/README.md +5 -0
  299. package/eval/langsmith/tasks/README.md +5 -0
  300. package/eval/langsmith/tests/conftest.py +187 -0
  301. package/eval/langsmith/tests/tasks/test_tasks.py +287 -0
  302. package/eval/langsmith/treatments/README.md +5 -0
  303. package/eval/local/README.md +80 -0
  304. package/eval/local/regression_baseline.json +13 -0
  305. package/eval/local/report-style-demo-assets/quality_cost.en.svg +41 -0
  306. package/eval/local/report-style-demo-assets/quality_cost.zh.svg +41 -0
  307. package/eval/local/report-style-demo-assets/rubric_delta.en.svg +56 -0
  308. package/eval/local/report-style-demo-assets/rubric_delta.zh.svg +56 -0
  309. package/eval/local/report-style-demo-assets/task_outcomes.en.svg +46 -0
  310. package/eval/local/report-style-demo-assets/task_outcomes.zh.svg +46 -0
  311. package/eval/local/report-style-demo.html +867 -0
  312. package/eval/local/scripts/compare_baselines.py +1018 -0
  313. package/eval/local/scripts/generate_report_style_demo_charts.py +322 -0
  314. package/eval/local/scripts/regression_check.py +171 -0
  315. package/eval/local/scripts/render_readme_benchmark_figures.py +416 -0
  316. package/eval/local/scripts/rescore_rubric.py +120 -0
  317. package/eval/local/skills/benchmarks/039-release/comet-classic-039/SKILL.md +282 -0
  318. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/auto-transition.md +27 -0
  319. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/comet-yaml-fields.md +68 -0
  320. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/context-recovery.md +35 -0
  321. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/debug-gate.md +17 -0
  322. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/decision-point.md +20 -0
  323. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/dirty-worktree.md +59 -0
  324. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/file-structure.md +28 -0
  325. package/eval/local/skills/benchmarks/039-release/comet-classic-039/reference/subagent-dispatch.md +113 -0
  326. package/eval/local/skills/benchmarks/039-release/comet-classic-039/rules/comet-phase-guard.md +125 -0
  327. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-archive.sh +311 -0
  328. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-env.sh +110 -0
  329. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-guard.sh +778 -0
  330. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-handoff.sh +390 -0
  331. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-hook-guard.sh +336 -0
  332. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-state.sh +1338 -0
  333. package/eval/local/skills/benchmarks/039-release/comet-classic-039/scripts/comet-yaml-validate.sh +225 -0
  334. package/eval/local/skills/benchmarks/039-release/comet-classic-039-archive/SKILL.md +100 -0
  335. package/eval/local/skills/benchmarks/039-release/comet-classic-039-build/SKILL.md +304 -0
  336. package/eval/local/skills/benchmarks/039-release/comet-classic-039-design/SKILL.md +263 -0
  337. package/eval/local/skills/benchmarks/039-release/comet-classic-039-hotfix/SKILL.md +204 -0
  338. package/eval/local/skills/benchmarks/039-release/comet-classic-039-open/SKILL.md +220 -0
  339. package/eval/local/skills/benchmarks/039-release/comet-classic-039-tweak/SKILL.md +180 -0
  340. package/eval/local/skills/benchmarks/039-release/comet-classic-039-verify/SKILL.md +234 -0
  341. package/eval/local/skills/benchmarks/040-beta/comet/SKILL.md +261 -0
  342. package/eval/local/skills/benchmarks/040-beta/comet/reference/auto-transition.md +27 -0
  343. package/eval/local/skills/benchmarks/040-beta/comet/reference/comet-yaml-fields.md +73 -0
  344. package/eval/local/skills/benchmarks/040-beta/comet/reference/context-recovery.md +45 -0
  345. package/eval/local/skills/benchmarks/040-beta/comet/reference/debug-gate.md +33 -0
  346. package/eval/local/skills/benchmarks/040-beta/comet/reference/decision-point.md +31 -0
  347. package/eval/local/skills/benchmarks/040-beta/comet/reference/dirty-worktree.md +59 -0
  348. package/eval/local/skills/benchmarks/040-beta/comet/reference/file-structure.md +28 -0
  349. package/eval/local/skills/benchmarks/040-beta/comet/reference/intent-frame.md +74 -0
  350. package/eval/local/skills/benchmarks/040-beta/comet/reference/scripts.md +69 -0
  351. package/eval/local/skills/benchmarks/040-beta/comet/reference/subagent-dispatch.md +169 -0
  352. package/eval/local/skills/benchmarks/040-beta/comet/rules/comet-phase-guard.md +130 -0
  353. package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/checks.yaml +6 -0
  354. package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/guardrails.yaml +13 -0
  355. package/eval/local/skills/benchmarks/040-beta/comet/runtime/classic/skill.yaml +122 -0
  356. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-archive.mjs +9515 -0
  357. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-env.mjs +15 -0
  358. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-guard.mjs +10173 -0
  359. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-handoff.mjs +9491 -0
  360. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-hook-guard.mjs +9175 -0
  361. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-intent.mjs +404 -0
  362. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-state.mjs +9273 -0
  363. package/eval/local/skills/benchmarks/040-beta/comet/scripts/comet-yaml-validate.mjs +7627 -0
  364. package/eval/local/skills/benchmarks/040-beta/comet-archive/SKILL.md +110 -0
  365. package/eval/local/skills/benchmarks/040-beta/comet-build/SKILL.md +315 -0
  366. package/eval/local/skills/benchmarks/040-beta/comet-design/SKILL.md +260 -0
  367. package/eval/local/skills/benchmarks/040-beta/comet-hotfix/SKILL.md +190 -0
  368. package/eval/local/skills/benchmarks/040-beta/comet-open/SKILL.md +210 -0
  369. package/eval/local/skills/benchmarks/040-beta/comet-tweak/SKILL.md +183 -0
  370. package/eval/local/skills/benchmarks/040-beta/comet-verify/SKILL.md +230 -0
  371. package/eval/local/skills/benchmarks/dependency/claude-md/comet-workflow/CLAUDE.md +12 -0
  372. package/eval/local/skills/benchmarks/dependency/openspec/openspec-apply-change/SKILL.md +156 -0
  373. package/eval/local/skills/benchmarks/dependency/openspec/openspec-archive-change/SKILL.md +114 -0
  374. package/eval/local/skills/benchmarks/dependency/openspec/openspec-bulk-archive-change/SKILL.md +246 -0
  375. package/eval/local/skills/benchmarks/dependency/openspec/openspec-continue-change/SKILL.md +118 -0
  376. package/eval/local/skills/benchmarks/dependency/openspec/openspec-explore/SKILL.md +288 -0
  377. package/eval/local/skills/benchmarks/dependency/openspec/openspec-ff-change/SKILL.md +101 -0
  378. package/eval/local/skills/benchmarks/dependency/openspec/openspec-new-change/SKILL.md +74 -0
  379. package/eval/local/skills/benchmarks/dependency/openspec/openspec-onboard/SKILL.md +554 -0
  380. package/eval/local/skills/benchmarks/dependency/openspec/openspec-propose/SKILL.md +110 -0
  381. package/eval/local/skills/benchmarks/dependency/openspec/openspec-sync-specs/SKILL.md +138 -0
  382. package/eval/local/skills/benchmarks/dependency/openspec/openspec-verify-change/SKILL.md +168 -0
  383. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/SKILL.md +159 -0
  384. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/frame-template.html +213 -0
  385. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/helper.js +167 -0
  386. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/server.cjs +723 -0
  387. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/start-server.sh +209 -0
  388. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/scripts/stop-server.sh +120 -0
  389. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/spec-document-reviewer-prompt.md +49 -0
  390. package/eval/local/skills/benchmarks/dependency/superpowers/brainstorming/visual-companion.md +291 -0
  391. package/eval/local/skills/benchmarks/dependency/superpowers/dispatching-parallel-agents/SKILL.md +185 -0
  392. package/eval/local/skills/benchmarks/dependency/superpowers/executing-plans/SKILL.md +70 -0
  393. package/eval/local/skills/benchmarks/dependency/superpowers/finishing-a-development-branch/SKILL.md +241 -0
  394. package/eval/local/skills/benchmarks/dependency/superpowers/receiving-code-review/SKILL.md +213 -0
  395. package/eval/local/skills/benchmarks/dependency/superpowers/requesting-code-review/SKILL.md +103 -0
  396. package/eval/local/skills/benchmarks/dependency/superpowers/requesting-code-review/code-reviewer.md +172 -0
  397. package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/SKILL.md +418 -0
  398. package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/implementer-prompt.md +139 -0
  399. package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/review-package +44 -0
  400. package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/sdd-workspace +22 -0
  401. package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/scripts/task-brief +40 -0
  402. package/eval/local/skills/benchmarks/dependency/superpowers/subagent-driven-development/task-reviewer-prompt.md +188 -0
  403. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/CREATION-LOG.md +119 -0
  404. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/SKILL.md +296 -0
  405. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/condition-based-waiting-example.ts +158 -0
  406. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/condition-based-waiting.md +115 -0
  407. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/defense-in-depth.md +122 -0
  408. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/find-polluter.sh +63 -0
  409. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/root-cause-tracing.md +169 -0
  410. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-academic.md +14 -0
  411. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-1.md +58 -0
  412. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-2.md +68 -0
  413. package/eval/local/skills/benchmarks/dependency/superpowers/systematic-debugging/test-pressure-3.md +69 -0
  414. package/eval/local/skills/benchmarks/dependency/superpowers/test-driven-development/SKILL.md +371 -0
  415. package/eval/local/skills/benchmarks/dependency/superpowers/test-driven-development/testing-anti-patterns.md +299 -0
  416. package/eval/local/skills/benchmarks/dependency/superpowers/using-git-worktrees/SKILL.md +202 -0
  417. package/eval/local/skills/benchmarks/dependency/superpowers/verification-before-completion/SKILL.md +139 -0
  418. package/eval/local/skills/benchmarks/dependency/superpowers/writing-plans/SKILL.md +174 -0
  419. package/eval/local/skills/benchmarks/dependency/superpowers/writing-plans/plan-document-reviewer-prompt.md +49 -0
  420. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/SKILL.md +689 -0
  421. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/anthropic-best-practices.md +1150 -0
  422. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/examples/CLAUDE_MD_TESTING.md +189 -0
  423. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/graphviz-conventions.dot +172 -0
  424. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/persuasion-principles.md +187 -0
  425. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/render-graphs.js +168 -0
  426. package/eval/local/skills/benchmarks/dependency/superpowers/writing-skills/testing-skills-with-subagents.md +384 -0
  427. package/eval/local/tasks/authoring-skill-smoke/environment/Dockerfile +2 -0
  428. package/eval/local/tasks/authoring-skill-smoke/instruction.md +4 -0
  429. package/eval/local/tasks/authoring-skill-smoke/task.toml +21 -0
  430. package/eval/local/tasks/authoring-skill-smoke/validation/test_authoring_skill_smoke.py +54 -0
  431. package/eval/local/tasks/comet-agent-memory-routing/environment/Dockerfile +28 -0
  432. package/eval/local/tasks/comet-agent-memory-routing/environment/agent_system.py +40 -0
  433. package/eval/local/tasks/comet-agent-memory-routing/environment/test_agent_system.py +29 -0
  434. package/eval/local/tasks/comet-agent-memory-routing/instruction.md +20 -0
  435. package/eval/local/tasks/comet-agent-memory-routing/task.toml +24 -0
  436. package/eval/local/tasks/comet-agent-memory-routing/validation/test_agent_memory_routing.py +24 -0
  437. package/eval/local/tasks/comet-api-cache-ttl/environment/Dockerfile +39 -0
  438. package/eval/local/tasks/comet-api-cache-ttl/environment/cache.py +39 -0
  439. package/eval/local/tasks/comet-api-cache-ttl/environment/test_cache.py +81 -0
  440. package/eval/local/tasks/comet-api-cache-ttl/instruction.md +33 -0
  441. package/eval/local/tasks/comet-api-cache-ttl/task.toml +24 -0
  442. package/eval/local/tasks/comet-api-cache-ttl/validation/test_api_cache_ttl.py +58 -0
  443. package/eval/local/tasks/comet-cross-file-refactor/environment/Dockerfile +15 -0
  444. package/eval/local/tasks/comet-cross-file-refactor/environment/cli.py +13 -0
  445. package/eval/local/tasks/comet-cross-file-refactor/environment/counting.py +13 -0
  446. package/eval/local/tasks/comet-cross-file-refactor/environment/test_textkit.py +21 -0
  447. package/eval/local/tasks/comet-cross-file-refactor/instruction.md +16 -0
  448. package/eval/local/tasks/comet-cross-file-refactor/task.toml +20 -0
  449. package/eval/local/tasks/comet-cross-file-refactor/validation/test_cross_file_refactor.py +20 -0
  450. package/eval/local/tasks/comet-dependency-confusion/environment/Dockerfile +28 -0
  451. package/eval/local/tasks/comet-dependency-confusion/environment/app.py +8 -0
  452. package/eval/local/tasks/comet-dependency-confusion/environment/settings_loader.py +14 -0
  453. package/eval/local/tasks/comet-dependency-confusion/environment/test_settings_service.py +19 -0
  454. package/eval/local/tasks/comet-dependency-confusion/instruction.md +18 -0
  455. package/eval/local/tasks/comet-dependency-confusion/task.toml +24 -0
  456. package/eval/local/tasks/comet-dependency-confusion/validation/test_dependency_confusion.py +24 -0
  457. package/eval/local/tasks/comet-fix-median/environment/Dockerfile +50 -0
  458. package/eval/local/tasks/comet-fix-median/environment/stats.py +54 -0
  459. package/eval/local/tasks/comet-fix-median/environment/test_stats.py +68 -0
  460. package/eval/local/tasks/comet-fix-median/instruction.md +25 -0
  461. package/eval/local/tasks/comet-fix-median/task.toml +24 -0
  462. package/eval/local/tasks/comet-fix-median/validation/test_fix_median.py +60 -0
  463. package/eval/local/tasks/comet-framework-selection/environment/Dockerfile +28 -0
  464. package/eval/local/tasks/comet-framework-selection/environment/architecture.py +12 -0
  465. package/eval/local/tasks/comet-framework-selection/environment/test_architecture.py +30 -0
  466. package/eval/local/tasks/comet-framework-selection/instruction.md +22 -0
  467. package/eval/local/tasks/comet-framework-selection/task.toml +24 -0
  468. package/eval/local/tasks/comet-framework-selection/validation/test_framework_selection.py +24 -0
  469. package/eval/local/tasks/comet-full-workflow/environment/Dockerfile +53 -0
  470. package/eval/local/tasks/comet-full-workflow/environment/test_wordcount.py +29 -0
  471. package/eval/local/tasks/comet-full-workflow/environment/wordcount.py +40 -0
  472. package/eval/local/tasks/comet-full-workflow/instruction.md +18 -0
  473. package/eval/local/tasks/comet-full-workflow/task.toml +24 -0
  474. package/eval/local/tasks/comet-full-workflow/validation/test_full_workflow.py +171 -0
  475. package/eval/local/tasks/comet-graph-execution-review/environment/Dockerfile +28 -0
  476. package/eval/local/tasks/comet-graph-execution-review/environment/pipeline.py +19 -0
  477. package/eval/local/tasks/comet-graph-execution-review/environment/test_pipeline.py +53 -0
  478. package/eval/local/tasks/comet-graph-execution-review/instruction.md +21 -0
  479. package/eval/local/tasks/comet-graph-execution-review/task.toml +24 -0
  480. package/eval/local/tasks/comet-graph-execution-review/validation/test_graph_execution_review.py +24 -0
  481. package/eval/local/tasks/comet-human-approval-flow/environment/Dockerfile +15 -0
  482. package/eval/local/tasks/comet-human-approval-flow/environment/approvals.py +17 -0
  483. package/eval/local/tasks/comet-human-approval-flow/environment/test_approvals.py +30 -0
  484. package/eval/local/tasks/comet-human-approval-flow/instruction.md +15 -0
  485. package/eval/local/tasks/comet-human-approval-flow/task.toml +20 -0
  486. package/eval/local/tasks/comet-human-approval-flow/validation/test_human_approval_flow.py +20 -0
  487. package/eval/local/tasks/comet-layered-streaming-fix/environment/Dockerfile +20 -0
  488. package/eval/local/tasks/comet-layered-streaming-fix/environment/chat_app.py +19 -0
  489. package/eval/local/tasks/comet-layered-streaming-fix/environment/test_chat_app.py +15 -0
  490. package/eval/local/tasks/comet-layered-streaming-fix/instruction.md +14 -0
  491. package/eval/local/tasks/comet-layered-streaming-fix/task.toml +20 -0
  492. package/eval/local/tasks/comet-layered-streaming-fix/validation/test_layered_streaming_fix.py +20 -0
  493. package/eval/local/tasks/comet-noise-distractor/environment/Dockerfile +15 -0
  494. package/eval/local/tasks/comet-noise-distractor/environment/distractor.md +4 -0
  495. package/eval/local/tasks/comet-noise-distractor/environment/invoice.py +5 -0
  496. package/eval/local/tasks/comet-noise-distractor/environment/test_invoice.py +18 -0
  497. package/eval/local/tasks/comet-noise-distractor/instruction.md +17 -0
  498. package/eval/local/tasks/comet-noise-distractor/task.toml +20 -0
  499. package/eval/local/tasks/comet-noise-distractor/validation/test_noise_distractor.py +20 -0
  500. package/eval/local/tasks/comet-observability-env-template/environment/Dockerfile +15 -0
  501. package/eval/local/tasks/comet-observability-env-template/environment/env_writer.py +17 -0
  502. package/eval/local/tasks/comet-observability-env-template/environment/test_env_writer.py +17 -0
  503. package/eval/local/tasks/comet-observability-env-template/instruction.md +20 -0
  504. package/eval/local/tasks/comet-observability-env-template/task.toml +20 -0
  505. package/eval/local/tasks/comet-observability-env-template/validation/test_observability_env_template.py +46 -0
  506. package/eval/local/tasks/comet-perf-dedupe/environment/Dockerfile +39 -0
  507. package/eval/local/tasks/comet-perf-dedupe/environment/dedupe.py +22 -0
  508. package/eval/local/tasks/comet-perf-dedupe/environment/test_dedupe.py +43 -0
  509. package/eval/local/tasks/comet-perf-dedupe/instruction.md +25 -0
  510. package/eval/local/tasks/comet-perf-dedupe/task.toml +24 -0
  511. package/eval/local/tasks/comet-perf-dedupe/validation/test_perf_dedupe.py +40 -0
  512. package/eval/local/tasks/comet-persistence-threading/environment/Dockerfile +15 -0
  513. package/eval/local/tasks/comet-persistence-threading/environment/session_store.py +15 -0
  514. package/eval/local/tasks/comet-persistence-threading/environment/test_session_store.py +21 -0
  515. package/eval/local/tasks/comet-persistence-threading/instruction.md +14 -0
  516. package/eval/local/tasks/comet-persistence-threading/task.toml +20 -0
  517. package/eval/local/tasks/comet-persistence-threading/validation/test_persistence_threading.py +20 -0
  518. package/eval/local/tasks/comet-refactor-counter/environment/Dockerfile +39 -0
  519. package/eval/local/tasks/comet-refactor-counter/environment/test_text_processor.py +57 -0
  520. package/eval/local/tasks/comet-refactor-counter/environment/text_processor.py +49 -0
  521. package/eval/local/tasks/comet-refactor-counter/instruction.md +31 -0
  522. package/eval/local/tasks/comet-refactor-counter/task.toml +24 -0
  523. package/eval/local/tasks/comet-refactor-counter/validation/test_refactor_counter.py +75 -0
  524. package/eval/local/tasks/comet-robust-config/environment/Dockerfile +39 -0
  525. package/eval/local/tasks/comet-robust-config/environment/config_loader.py +56 -0
  526. package/eval/local/tasks/comet-robust-config/environment/test_config_loader.py +72 -0
  527. package/eval/local/tasks/comet-robust-config/instruction.md +30 -0
  528. package/eval/local/tasks/comet-robust-config/task.toml +24 -0
  529. package/eval/local/tasks/comet-robust-config/validation/test_robust_config.py +37 -0
  530. package/eval/local/tasks/generic-skill-smoke/environment/Dockerfile +2 -0
  531. package/eval/local/tasks/generic-skill-smoke/instruction.md +7 -0
  532. package/eval/local/tasks/generic-skill-smoke/task.toml +21 -0
  533. package/eval/local/tasks/generic-skill-smoke/validation/test_generic_skill_smoke.py +36 -0
  534. package/eval/local/tasks/index.yaml +155 -0
  535. package/eval/local/tasks/workflow-overlay-contract/environment/Dockerfile +5 -0
  536. package/eval/local/tasks/workflow-overlay-contract/instruction.md +5 -0
  537. package/eval/local/tasks/workflow-overlay-contract/task.toml +21 -0
  538. package/eval/local/tasks/workflow-overlay-contract/validation/test_workflow_overlay_contract.py +235 -0
  539. package/eval/local/tasks/workflow-route-conformance/environment/Dockerfile +2 -0
  540. package/eval/local/tasks/workflow-route-conformance/instruction.md +5 -0
  541. package/eval/local/tasks/workflow-route-conformance/task.toml +21 -0
  542. package/eval/local/tasks/workflow-route-conformance/validation/test_workflow_route_conformance.py +84 -0
  543. package/eval/local/tests/conftest.py +1148 -0
  544. package/eval/local/tests/scaffold/test_attribution.py +48 -0
  545. package/eval/local/tests/scaffold/test_compare_baselines.py +999 -0
  546. package/eval/local/tests/scaffold/test_conftest_helpers.py +534 -0
  547. package/eval/local/tests/scaffold/test_evidence.py +15 -0
  548. package/eval/local/tests/scaffold/test_langsmith_conftest.py +244 -0
  549. package/eval/local/tests/scaffold/test_logging.py +243 -0
  550. package/eval/local/tests/scaffold/test_manifests.py +132 -0
  551. package/eval/local/tests/scaffold/test_profiles.py +866 -0
  552. package/eval/local/tests/scaffold/test_regression_check.py +19 -0
  553. package/eval/local/tests/scaffold/test_report_style_demo_charts.py +45 -0
  554. package/eval/local/tests/scaffold/test_sample_quality.py +211 -0
  555. package/eval/local/tests/scaffold/test_tasks.py +168 -0
  556. package/eval/local/tests/scaffold/test_treatments.py +244 -0
  557. package/eval/local/tests/scaffold/test_utils.py +139 -0
  558. package/eval/local/tests/tasks/test_tasks.py +445 -0
  559. package/eval/local/tests/tasks/test_validation_scripts.py +143 -0
  560. package/eval/local/treatments/comet/comet_full_039.yaml +141 -0
  561. package/eval/local/treatments/comet/comet_full_040_beta.yaml +140 -0
  562. package/eval/local/treatments/common/control.yaml +4 -0
  563. package/eval/pyproject.toml +41 -0
  564. package/eval/report-html-config.json +6 -0
  565. package/eval/scaffold/__init__.py +67 -0
  566. package/eval/scaffold/python/__init__.py +95 -0
  567. package/eval/scaffold/python/attribution.py +43 -0
  568. package/eval/scaffold/python/evidence.py +38 -0
  569. package/eval/scaffold/python/external_data_handler.py +18 -0
  570. package/eval/scaffold/python/generic_llm_judge.py +235 -0
  571. package/eval/scaffold/python/judge_config.py +168 -0
  572. package/eval/scaffold/python/llm_judge.py +191 -0
  573. package/eval/scaffold/python/logging.py +705 -0
  574. package/eval/scaffold/python/manifests.py +129 -0
  575. package/eval/scaffold/python/paper_charts.py +25 -0
  576. package/eval/scaffold/python/pass_at_k.py +107 -0
  577. package/eval/scaffold/python/paths.py +56 -0
  578. package/eval/scaffold/python/profiles.py +127 -0
  579. package/eval/scaffold/python/report_outputs.py +1391 -0
  580. package/eval/scaffold/python/sample_quality.py +339 -0
  581. package/eval/scaffold/python/schema.py +39 -0
  582. package/eval/scaffold/python/skill_parser.py +469 -0
  583. package/eval/scaffold/python/tasks.py +330 -0
  584. package/eval/scaffold/python/treatments.py +271 -0
  585. package/eval/scaffold/python/utils.py +366 -0
  586. package/eval/scaffold/python/validation/__init__.py +21 -0
  587. package/eval/scaffold/python/validation/authoring_rubric.py +326 -0
  588. package/eval/scaffold/python/validation/comet_workflow.py +187 -0
  589. package/eval/scaffold/python/validation/core.py +264 -0
  590. package/eval/scaffold/python/validation/dataset.py +337 -0
  591. package/eval/scaffold/python/validation/docker.py +106 -0
  592. package/eval/scaffold/python/validation/evaluator.py +549 -0
  593. package/eval/scaffold/python/validation/generic_rubric.py +169 -0
  594. package/eval/scaffold/python/validation/rubric.py +740 -0
  595. package/eval/scaffold/python/validation/runner.py +237 -0
  596. package/eval/scaffold/python/validation/scripts.py +58 -0
  597. package/eval/scaffold/python/validation/tracing.py +313 -0
  598. package/eval/scaffold/shell/common.sh +126 -0
  599. package/eval/scaffold/shell/docker.sh +482 -0
  600. package/eval/scaffold/shell/run-claude-loop.sh +181 -0
  601. package/eval/scaffold/shell/setup.sh +259 -0
  602. package/eval/simulator-instruction.md +9 -0
  603. package/package.json +29 -2
  604. package/dist/domains/dashboard/web/assets/index-BcO3_QlT.css +0 -1
  605. package/dist/domains/dashboard/web/assets/index-CjU3VXVu.js +0 -22
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: comet-design
3
- description: "Use when a full Comet change has completed open but lacks a Superpowers Design Doc, or design must resume from an OpenSpec handoff package."
3
+ description: "Use only when explicitly invoked as /comet-design or routed by the root Comet skill/runtime to a full workflow design phase; create or recover the deep technical Design Doc."
4
4
  ---
5
5
 
6
6
  # Comet Phase 2: Deep Design (Design)
@@ -19,7 +19,8 @@ description: "Use when a full Comet change has completed open but lacks a Superp
19
19
  Locate scripts via `comet/reference/scripts.md`, then run entry verification. When resuming from any entry point, first run the recovery check in `comet/reference/context-recovery.md`:
20
20
 
21
21
  ```bash
22
- node "$COMET_STATE" check <name> design
22
+ comet state select <change-name>
23
+ comet state check <name> design
23
24
  ```
24
25
 
25
26
  Proceed to Step 1 after verification passes. The script outputs specific failure reasons when verification fails.
@@ -31,7 +32,7 @@ Proceed to Step 1 after verification passes. The script outputs specific failure
31
32
  **Must be generated by script. Agent writing summaries on the fly is not allowed.**
32
33
 
33
34
  ```bash
34
- node "$COMET_HANDOFF" <change-name> design --write
35
+ comet handoff <change-name> design --write
35
36
  ```
36
37
 
37
38
  The script reads the change `.comet.yaml` `context_compression` snapshot, then generates and records the matching handoff package.
@@ -70,7 +71,7 @@ The beta handoff package is a **structured spec projection** that reduces OpenSp
70
71
  If full context is genuinely needed, explicitly run:
71
72
 
72
73
  ```bash
73
- node "$COMET_HANDOFF" <change-name> design --write --full
74
+ comet handoff <change-name> design --write --full
74
75
  ```
75
76
 
76
77
  Handoff package sources come from OpenSpec open phase artifacts:
@@ -86,7 +87,7 @@ Handoff package sources come from OpenSpec open phase artifacts:
86
87
  When loading the skill, ARGUMENTS must include:
87
88
 
88
89
  ```text
89
- Language: Use the configured Comet artifact language from `"$COMET_BASH" "$COMET_STATE" get <name> language`
90
+ Language: Use the configured Comet artifact language from `comet state get <name> language`
90
91
  ```
91
92
 
92
93
  After the skill loads, follow its guidance and use the following context:
@@ -145,9 +146,7 @@ Only after the user explicitly confirms, proceed to Step 2. If the user requests
145
146
 
146
147
  After the user confirms the design proposal, before creating the Design Doc, create or update the incrementally maintained checkpoint file and finalize it as the confirmed design summary:
147
148
 
148
- ```bash
149
- mkdir -p openspec/changes/<name>/.comet/handoff
150
- ```
149
+ Use the current platform's file API to ensure `openspec/changes/<name>/.comet/handoff/` exists; do not rely on a POSIX-only directory command.
151
150
 
152
151
  `openspec/changes/<name>/.comet/handoff/brainstorm-summary.md` structure:
153
152
 
@@ -179,14 +178,9 @@ mkdir -p openspec/changes/<name>/.comet/handoff
179
178
  - `openspec/changes/<name>/.comet/handoff/design-context.md` (or `spec-context.md` in beta mode)
180
179
  - `openspec/changes/<name>/.comet/handoff/design-context.json` (or `spec-context.json` in beta mode)
181
180
 
182
- ### 1e. Active Context Compaction Gate
183
-
184
- After Step 1d completes and `brainstorm-summary.md` is written, enter the active compaction gate before creating the Design Doc. At this point the OpenSpec handoff, brainstorming decisions, and pending items are durable, so the agent should proactively release the earlier Spec and brainstorming context to preserve window space for Step 2 and the later Build phase.
181
+ ### 1e. Compaction Policy (Non-blocking Here)
185
182
 
186
- Rules:
187
- - If the current platform provides a native context compaction/cleanup mechanism (for example, the host agent's compact/compaction command, tool, or UI action), trigger active compaction here once; do not try to fake compaction through a shell script.
188
- - The compaction resume prompt must include the change name, current step (Design Step 2), and the three handoff file categories listed above.
189
- - If the current platform cannot be compacted programmatically by the agent, pause and tell the user to run the host platform's manual compaction action; continue to Step 2 only after the user confirms compaction is unavailable or asks to continue.
183
+ `brainstorm-summary.md` is a recovery checkpoint, but do not discard live design context before the Design Doc is persisted. Continue directly to Step 2. Active compaction moves until after the Design Doc, state, and latest handoff are all durable.
190
184
 
191
185
  ### 2. Create Design Doc
192
186
 
@@ -213,17 +207,25 @@ First record the design_doc path. If Spec Patches wrote back delta spec (added o
213
207
 
214
208
  ```bash
215
209
  # Record design_doc path
216
- node "$COMET_STATE" set <name> design_doc docs/superpowers/specs/YYYY-MM-DD-topic-design.md
210
+ comet state set <name> design_doc docs/superpowers/specs/YYYY-MM-DD-topic-design.md
217
211
 
218
212
  # If delta spec changes exist, regenerate handoff (update hash)
219
- node "$COMET_HANDOFF" <change-name> design --write
213
+ comet handoff <change-name> design --write
220
214
 
221
215
  # Auto-transition to next phase
222
- node "$COMET_GUARD" <change-name> design --apply
216
+ comet guard <change-name> design --apply
223
217
  ```
224
218
 
225
219
  If there are no delta spec changes, skip the handoff regeneration step. The state file updates automatically; no manual editing of other fields needed.
226
220
 
221
+ ### 3a. Optional Active Context Compaction
222
+
223
+ Consider active compaction only **after the Design Doc and state evidence are persisted** and before Build. First confirm `design_doc`, the latest handoff, `handoff_hash`, and the design guard are durable so recovery never depends on an unwritten design judgment.
224
+
225
+ - If the host exposes a programmatically callable native compaction mechanism and the context window is under pressure, invoke it once and include the change, next step, Design Doc, and handoff files in the resume prompt
226
+ - If only the user can trigger compaction, give one non-blocking suggestion and continue; it **must not block when programmatic compaction is unavailable** and must not create another confirmation point
227
+ - Never fake host compaction with a shell command or an agent summary
228
+
227
229
  ## Exit Conditions
228
230
 
229
231
  - Design Doc created and saved
@@ -234,12 +236,12 @@ If there are no delta spec changes, skip the handoff regeneration step. The stat
234
236
  - In beta mode, `spec-context.json` must be structurally valid and reference the current source files (enforced by guard)
235
237
  - If new capabilities or supplementary acceptance scenarios exist, OpenSpec delta spec has been created/updated
236
238
  - `design_doc` written to `.comet.yaml`
237
- - **Phase guard**: Run `node "$COMET_GUARD" <change-name> design --apply`; after all PASS, auto-transitions to `phase: build`
239
+ - **Phase guard**: Run `comet guard <change-name> design --apply`; after all PASS, auto-transitions to `phase: build`
238
240
 
239
241
  Must use `--apply` before exit:
240
242
 
241
243
  ```bash
242
- node "$COMET_GUARD" <change-name> design --apply
244
+ comet guard <change-name> design --apply
243
245
  ```
244
246
 
245
247
  ## Context Compression Recovery
@@ -251,9 +253,9 @@ Follow `comet/reference/context-recovery.md` with phase set to `design`.
251
253
  Follow `comet/reference/auto-transition.md`. Key command:
252
254
 
253
255
  ```bash
254
- node "$COMET_STATE" next <change-name>
256
+ comet state next <change-name>
255
257
  ```
256
258
 
257
259
  - `NEXT: auto` → invoke the skill pointed to by `SKILL` to enter the next phase
258
- - `NEXT: manual` → do not invoke the next skill; prompt user to run `/<SKILL>` manually
260
+ - `NEXT: manual` → do not invoke the next skill; return control with `HINT`, end the invocation, and do not create another confirmation point
259
261
  - `NEXT: done` → workflow is complete, no further action needed
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: comet-hotfix
3
- description: "Use when the user wants to fix an existing behavior bug without adding capability or needing full design; also use when resuming hotfix workflow."
3
+ description: "Use only when explicitly invoked as /comet-hotfix or routed by the root Comet skill/runtime to the hotfix preset; fix an existing behavior bug, not an ordinary unmanaged bugfix."
4
4
  ---
5
5
 
6
6
  # Comet Preset Path: Hotfix
@@ -20,64 +20,68 @@ Quick bug fix workflow: open → build → verify → archive. Skip brainstormin
20
20
 
21
21
  ### 0. Output Language Constraint
22
22
 
23
- Streamlined OpenSpec artifacts must use the configured Comet artifact language. Read `language` from `.comet/config.yaml` before `.comet.yaml` exists, then use `"$COMET_BASH" "$COMET_STATE" get <name> language` after initialization.
23
+ Streamlined OpenSpec artifacts must use the configured Comet artifact language. Before `.comet.yaml` exists, read `language` from project `.comet/config.yaml`, then fall back to global `~/.comet/config.yaml`; after initialization, use `comet state get <name> language`.
24
24
 
25
25
  Execution chain: open → build → root cause check → verify → archive. Hotfix provides default decisions for each phase: streamlined open, direct build, root cause confirmation, scale-based verification, and final archive confirmation after verification passes.
26
26
 
27
27
  Before starting, locate Comet scripts via `comet/reference/scripts.md`. When resuming from any entry point, first use `comet/reference/context-recovery.md` to confirm phase/workflow.
28
28
 
29
+ When resuming an existing hotfix change, the first state operation must be `comet state select <change-name>`. For a new change, run the command immediately after `.comet.yaml` initialization and before source writes.
30
+
29
31
  ### 1. Quick Open (preset open)
30
32
 
31
33
  Reuse Comet open capability to create change, but use hotfix defaults: do not execute `openspec-explore` long exploration, directly enter streamlined change creation.
32
34
 
33
35
  **Immediately execute:** Use the Skill tool to load the `openspec-new-change` skill. Skipping this step is prohibited.
34
36
 
35
- After the skill loads, follow its guidance to create streamlined artifacts:
36
- - `proposal.md` — problem description + root cause analysis + fix goal (no solution comparison needed)
37
- - `design.md` — fix solution (one is enough, no multi-solution comparison needed)
38
- - `tasks.md` — fix task list
39
- - **No delta spec needed** (unless fix changes existing spec acceptance scenarios)
40
-
41
- Initialize Comet state file:
37
+ After the skill loads, create the change skeleton first, then immediately initialize recoverable state and bind the current change:
42
38
 
43
39
  ```bash
44
- node "$COMET_STATE" init <name> hotfix
40
+ comet state init <name> hotfix
41
+ comet state select <name>
42
+ comet state check <name> open
45
43
  ```
46
44
 
47
- Verify initialized state:
48
-
49
- ```bash
50
- node "$COMET_STATE" check <name> open
51
- ```
45
+ Hotfix defaults to `isolation: current`, truthfully indicating execution in the current workspace. Change it to `branch` or `worktree` only after that workspace is actually created/selected. Then create the streamlined artifacts:
46
+ - `proposal.md` — problem description + root cause analysis + fix goal (no solution comparison needed)
47
+ - `design.md` — fix solution (one is enough, no multi-solution comparison needed)
48
+ - `tasks.md` fix task list
49
+ - **No delta spec needed** (unless fix changes existing spec acceptance scenarios)
52
50
 
53
51
  Run phase guard to transition open → build:
54
52
 
55
53
  ```bash
56
- node "$COMET_GUARD" <change-name> open --apply
54
+ comet guard <change-name> open --apply
57
55
  ```
58
56
 
59
57
  Check `auto_transition` to decide whether to continue:
60
58
 
61
59
  ```bash
62
- node "$COMET_STATE" next <name>
60
+ comet state next <name>
63
61
  ```
64
62
 
65
63
  - `NEXT: auto` → continue to Step 2
66
- - `NEXT: manual` → pause, follow `HINT` to prompt user to run `/<SKILL>` manually
64
+ - `NEXT: manual` → return control with `HINT` and end the current invocation; do not ask whether to continue
67
65
 
68
66
  ### 2. Direct Build (preset build)
69
67
 
70
- Use hotfix defaults: `build_mode: direct`, `review_mode: off` (hotfix/tweak skip review_mode selection the guard does not require it for preset workflows). Skip Superpowers `brainstorming` and `writing-plans` (unless tasks > 3; if exceeds 3 tasks, transfer to `/comet-build`'s plan and execution method selection note this does NOT trigger full workflow upgrade, only switches execution method).
68
+ Use hotfix defaults: `build_mode: direct`, `tdd_mode: direct`, `review_mode: off`, and `isolation: current`. Here `direct` skips full planning/TDD orchestration; it never skips reproduction, regression coverage, or verification. Skip Superpowers `brainstorming` and `writing-plans`; **task count alone does not route to `/comet-build`**. Keep larger task lists ordered in the current hotfix and ask about upgrading only when a qualitative-change signal or scope tripwire is hit.
71
69
 
72
70
  Before continuing or starting changes, handle uncommitted changes through `comet/reference/dirty-worktree.md`. If attribution shows a qualitative-change signal or file-count tripwire is hit, handle it through this file's "Upgrade Assessment".
73
71
 
74
- **Immediately execute:** Execute tasks one by one according to tasks.md:
72
+ Before implementation, **reproduce the bug and record failing evidence first**:
73
+
74
+ 1. Confirm the reported old behavior with minimal repeatable steps and record the command, input, and actual result
75
+ 2. When automatable, add and run a regression test that fails for this bug; confirm the failure is caused by the bug rather than the environment or test itself
76
+ 3. If automation is temporarily impossible, record why plus repeatable manual failing evidence in the proposal/verification report; never edit code without evidence
77
+
78
+ After RED evidence exists, execute tasks one by one according to tasks.md:
75
79
 
76
80
  1. Read `openspec/changes/<name>/tasks.md`, get incomplete task list
77
81
  2. For each incomplete task:
78
82
  - Modify code according to task description
79
83
  - Run project formatter (e.g., `mvn spotless:apply`, `npm run format`)
80
- - Run related tests to confirm pass
84
+ - First rerun the new failing regression test and confirm it turns green, then run related tests
81
85
  - Check corresponding `- [ ]` to `- [x]` in tasks.md
82
86
  - Commit code, commit message format: `fix: <brief fix description>`
83
87
  3. After all tasks complete, explicitly run relevant project tests and build commands
@@ -105,7 +109,7 @@ For specific investigation, minimal failing test, fix verification, and keeping
105
109
  After root cause is confirmed eliminated, run phase guard to transition build → verify:
106
110
 
107
111
  ```bash
108
- node "$COMET_GUARD" <change-name> build --apply
112
+ comet guard <change-name> build --apply
109
113
  ```
110
114
 
111
115
  State automatically updates to `phase: verify`, `verify_result: pending`, then enter verification.
@@ -116,7 +120,7 @@ Reuse `/comet-verify`, with comet-verify's scale assessment deciding lightweight
116
120
 
117
121
  **Immediately execute:** Use the Skill tool to load the `comet-verify` skill. Skipping this step is prohibited.
118
122
 
119
- Small-scale hotfixes without delta spec usually meet lightweight verification conditions (≤ 3 tasks, changed files below the scale threshold), comet-verify's scale assessment will select the lightweight verification path (6 quick checks; default `review_mode: off` does not dispatch automatic code review). If the user wants to increase review, they can run `node "$COMET_STATE" set <name> review_mode standard` or `thorough` before verification. If hotfix created delta spec, enter full verification path according to comet-verify's scale assessment rules.
123
+ Small-scale hotfixes without delta spec usually meet lightweight verification conditions (≤ 3 tasks, changed files below the scale threshold), comet-verify's scale assessment will select the lightweight verification path (6 quick checks; default `review_mode: off` does not dispatch automatic code review). If the user wants to increase review, they can run `comet state set <name> review_mode standard` or `thorough` before verification. If hotfix created delta spec, enter full verification path according to comet-verify's scale assessment rules.
120
124
 
121
125
  After verification passes, record `.comet.yaml` `verify_result` as `pass` according to `/comet-verify` rules, must not skip this status before archiving. After verification passes, still enter `/comet-archive`'s final archive confirmation; do not automatically run the archive script.
122
126
 
@@ -134,14 +138,13 @@ If there is delta spec, sync to main spec according to comet-archive rules, and
134
138
  <IMPORTANT>
135
139
  Hotfix workflow is **one-time continuous execution**. After invoking `/comet-hotfix`, agent must automatically advance through hotfix steps, without pausing to wait for user input mid-way.
136
140
 
137
- Exception: when `.comet.yaml` has `auto_transition: false`, after each phase guard advances `phase`, do not auto-invoke the next skill. In this case, use `node "$COMET_STATE" next <name>` output and pause for manual continuation as instructed.
141
+ Exception: when `.comet.yaml` has `auto_transition: false`, end the current invocation at each phase boundary and return control with `HINT`; the user may run the next phase later. This is a manual handoff, not a new confirmation point.
138
142
 
139
- The following situations must also pause and wait for user confirmation:
143
+ The following genuine user decisions still pause:
140
144
 
141
145
  1. Encountering an upgrade-assessment signal (see "Upgrade Assessment" section). **Must use the current platform's available user input/confirmation mechanism to pause and wait for the user to explicitly choose**: continue the hotfix flow, or upgrade to the full `/comet` workflow
142
- 2. workspace isolation and execution-method selection when tasks exceed 3 and transfer to `/comet-build`
143
- 3. verify phase (comet-verify) verification-failure and branch-handling decisions
144
- 4. Final archive confirmation (before comet-archive runs the archive script)
146
+ 2. Verify-phase acceptance of WARNING/SUGGESTION deviations, Spec drift handling, or strategy after the automatic repair limit; the first 3 clearly repairable failures close automatically
147
+ 3. Final archive confirmation and the branch-handling decision after the archive commit
145
148
 
146
149
  Execution order: quick open → direct build → root cause check → verification → archive → complete
147
150
 
@@ -152,7 +155,7 @@ After each step completes, immediately enter next step. Within each phase, must
152
155
 
153
156
  ## Upgrade Assessment
154
157
 
155
- Hotfix upgrade assessment only decides whether to move from the preset workflow to full; file count never upgrades automatically, and `comet-state scale` only decides verification weight.
158
+ Hotfix upgrade assessment only decides whether to move from the preset workflow to full; file count never upgrades automatically, and `comet state scale` only decides verification weight.
156
159
 
157
160
  If `/comet` passes an intent frame from the entry, hotfix must recheck `risk_signal` and escalation signals only before build: new capability, public API, schema change, cross-module coordination, or deep architecture work. When any signal matches, enter the existing escalation decision point; do not reimplement entry intent recognition.
158
161
 
@@ -165,10 +168,10 @@ When a qualitative-change signal or file-count tripwire is hit, **must pause und
165
168
  After the user chooses upgrade (option B), use the legal state-machine upgrade channel, a single command that converts the preset workflow to full and rolls back to design:
166
169
 
167
170
  ```bash
168
- node "$COMET_STATE" transition <name> preset-escalate
171
+ comet state transition <name> preset-escalate
169
172
  ```
170
173
 
171
- This command atomically sets `workflow`/`classic_profile` to `full`, rolls `phase` back to `design`, and clears `design_doc` (satisfying comet-design entry requirements). Then add the Design Doc on the current change: **immediately use the Skill tool to load the `comet-design` skill**, then proceed through the normal full workflow.
174
+ This command atomically sets `workflow`/`classic_profile` to `full`, rolls `phase` back to `design`, clears `design_doc`, and clears preset-only `build_mode`, `tdd_mode`, `review_mode`, `isolation`, and `verify_mode`. Then add the Design Doc on the current change: **immediately use the Skill tool to load the `comet-design` skill**. On entering build, run the full joint workflow-configuration decision again.
172
175
 
173
176
  When the user chooses continue (option A), continue the hotfix workflow and record the user's reason for continuing.
174
177
 
@@ -179,16 +182,16 @@ When the user chooses continue (option A), continue the hotfix workflow and reco
179
182
  - Bug fixed, tests pass
180
183
  - Change archived
181
184
  - If spec changes, synced to main spec
182
- - **Phase guard**: Before build → verify run `node "$COMET_GUARD" <change-name> build --apply`; before verify → archive follow `/comet-verify` and run `node "$COMET_GUARD" <change-name> verify --apply`
185
+ - **Phase guard**: Before build → verify run `comet guard <change-name> build --apply`; before verify → archive follow `/comet-verify` and run `comet guard <change-name> verify --apply`
183
186
 
184
187
  ## Automatic Handoff to Next Phase
185
188
 
186
189
  Follow `comet/reference/auto-transition.md`. Key command:
187
190
 
188
191
  ```bash
189
- node "$COMET_STATE" next <name>
192
+ comet state next <name>
190
193
  ```
191
194
 
192
195
  - `NEXT: auto` → invoke the skill pointed to by `SKILL` to continue hotfix workflow (`phase: build` returns `comet-hotfix`, `verify` returns `comet-verify`, `archive` returns `comet-archive`)
193
- - `NEXT: manual` → do not invoke the next skill; prompt user to manually run `/<SKILL>` per `HINT`
196
+ - `NEXT: manual` → do not invoke the next skill; return control with `HINT`, end the invocation, and do not create another confirmation point
194
197
  - `NEXT: done` → workflow is complete, no further action needed
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: comet-open
3
- description: "Use when Comet needs to create a new OpenSpec change, or an active change is missing proposal/design/tasks/.comet.yaml initialization artifacts."
3
+ description: "Use only when explicitly invoked as /comet-open or routed by the root Comet skill/runtime to the open phase; create or recover an OpenSpec change and its proposal/design/tasks/.comet.yaml artifacts."
4
4
  ---
5
5
 
6
6
  # Comet Phase 1: Open
@@ -13,7 +13,31 @@ description: "Use when Comet needs to create a new OpenSpec change, or an active
13
13
 
14
14
  ### 0. Output Language Constraint
15
15
 
16
- Every prompt and artifact request passed to OpenSpec must include the resolved Comet artifact language, using normalized ids such as `en` or `zh-CN`. Before `.comet.yaml` exists, read `language` from `.comet/config.yaml`; after the change is initialized, use `"$COMET_BASH" "$COMET_STATE" get <name> language`. If no configured language exists, fall back to the current user request language. The generated `proposal.md`, `design.md`, and `tasks.md` must use that language as their main language.
16
+ Every prompt and artifact request passed to OpenSpec must include the resolved Comet artifact language, using normalized ids such as `en` or `zh-CN`. Before `.comet.yaml` exists, read `language` from project `.comet/config.yaml`, then fall back to global `~/.comet/config.yaml`; after the change is initialized, use `comet state get <name> language`. If no configured language exists, fall back to the current user request language. The generated `proposal.md`, `design.md`, and `tasks.md` must use that language as their main language.
17
+
18
+ ### 0a. Current Change Binding
19
+
20
+ When resuming an existing change, inspect `openspec/changes/<change-name>/.comet.yaml` first:
21
+
22
+ - If it exists and parses, select the change as the first state operation
23
+ - If it is missing but the change directory is valid, run `comet state init <change-name> full`, then select the change
24
+ - If it is malformed, stop and report the parse error; repair it manually from version control, a backup, or verifiable artifacts before continuing, and never overwrite a damaged file with `state set`
25
+
26
+ ```bash
27
+ comet state select <change-name>
28
+ ```
29
+
30
+ When creating a new change, initialize `.comet.yaml` first, then immediately run the same command; never fabricate a selection before state exists.
31
+
32
+ ### 0b. OpenSpec Compatibility Check
33
+
34
+ Before any OpenSpec status or instructions command, run:
35
+
36
+ ```bash
37
+ openspec --version
38
+ ```
39
+
40
+ This flow requires **OpenSpec >= 1.5.0**. Stop immediately if the version is older than 1.5.0, cannot be parsed, the command is unavailable, or it exits non-zero. Ask the user to run `npm install -g @fission-ai/openspec@latest` and retry. Never continue with an older CLI that lacks the `applyRequires`, `artifactPaths`, `changeRoot`, or `resolvedOutputPath` contracts.
17
41
 
18
42
  ### 1. Explore Ideas and Clarify Requirements
19
43
 
@@ -57,36 +81,44 @@ Every accepted split item must be created as an independent change through `/com
57
81
 
58
82
  Must not create proposal.md, design.md, or tasks.md before the user completes the PRD split choice. If the user chooses to create multiple changes, the current `/comet-open` invocation only completes split confirmation and coordination, then enters `/comet-open` for each split item in the user-confirmed order.
59
83
 
84
+ Immediately after the user confirms multiple changes, persist the accepted split to `.comet/batches/<batch-id>.json`. Use a stable kebab-case `batch-id`. The file must record at least `version`, the original goal summary, creation time, the ordered change names, and each item's goals, scope, non-goals, acceptance scenarios, and `pending|open-complete|selected` status. Atomically update it after each item is created or completed. This is a batch orchestration manifest, not a replacement for each change's `.comet.yaml`.
85
+
60
86
  In batch split mode, entering `/comet-open` for each split item must explicitly mark it as a "confirmed split item" and carry that split item's goals, scope, non-goals, and acceptance scenarios. Confirmed split items skip the PRD split preflight by default, unless the split item itself still clearly contains multiple independent capabilities.
61
87
 
62
- In batch split mode, a single split item must not auto-advance to `/comet-design` after completing the open phase. After splitting is complete, must pause and ask the user which change to start; after the user chooses, advance only that change into `/comet-design`, while other changes remain active and can be resumed later through `/comet`.
88
+ In batch split mode, a single split item must not auto-advance to `/comet-design` after completing the open phase.
63
89
 
64
- Minimal resume rule: do not add a dedicated batch state file. On resume, first check already-created active changes; split items that already exist and contain `.comet.yaml` must not be created again, while uncreated split items continue through `/comet-open` according to the user-confirmed split list. If the confirmed split list cannot be recovered from the conversation, must ask the user to confirm the split list again before continuing.
90
+ **Batch completion hard check (must not be skipped)**: after every split item completes its own open phase, run the following for each `<name>` in the user-confirmed list:
65
91
 
66
- ### 1b. Requirements Clarification Completion Confirmation (Blocking Point)
92
+ ```bash
93
+ openspec status --change "<name>" --json
94
+ comet state check <name> design
95
+ ```
67
96
 
68
- Before creating OpenSpec artifacts, must follow the `comet/reference/decision-point.md` protocol to pause and wait for the user to confirm requirements clarification is complete.
97
+ The OpenSpec JSON must satisfy all of these conditions:
98
+ - Resolved `changeRoot` must equal repository-local `openspec/changes/<name>`; stop if it does not, because Classic runtime does not support an external change root
99
+ - The schema must include core artifact ids `proposal`, `design`, and `tasks`; extra artifacts are allowed, but a missing core id is an incompatible schema
100
+ - Every artifact listed in `applyRequires` must be `done` in `artifacts`
101
+ - Concrete outputs in `artifactPaths.<artifact-id>.existingOutputPaths` (or `resolvedOutputPath` from instructions) must exist and be non-empty
102
+ - Treat `isComplete` as diagnostic only; it neither replaces the `applyRequires` implementation-readiness check nor lets optional artifacts block phase advancement
69
103
 
70
- When pausing, present the clarification summary: goals, non-goals, scope boundaries, key unknowns, and draft acceptance scenarios.
104
+ If any split item fails these checks, must not report splitting complete or ask which change to start. Stop and resume `/comet-open` from that change's first `ready` or `blocked` artifact. If OpenSpec passes but Comet state fails, repair `.comet.yaml` initialization or phase, then rerun the checks for the entire batch.
71
105
 
72
- Must not create proposal.md, design.md, or tasks.md before the user confirms requirements clarification is complete, and must not use the Skill tool to load the `openspec-propose` skill to generate all artifacts in one pass.
106
+ Only after every split item passes both CLI checks may you pause and ask which change to start. Mark the chosen item `selected` in the batch manifest, then advance only that change into `/comet-design`; other changes remain active and can be resumed later through `/comet`.
73
107
 
74
- ### 1c. Change Name Confirmation (Blocking Point)
108
+ On resume, read `.comet/batches/<batch-id>.json` first, then run the CLI checks above for already-created active changes. Do not recreate items that fully pass; resume incomplete items from the first `ready` artifact returned by OpenSpec. Create missing items from the persisted manifest. If the manifest is missing or damaged, stop and ask the user to rebuild/confirm it instead of inferring the original batch boundary from directory names.
75
109
 
76
- Before creating the change directory (`openspec new change`), must follow the `comet/reference/decision-point.md` protocol to pause and let the user decide the change name. Must not auto-generate or silently infer the change name.
110
+ ### 1b. Resolve Requirements and Change Name (Non-blocking by Default)
77
111
 
78
- OpenSpec change names must be **kebab-case English** (lowercase letters, digits, hyphens; e.g. `refine-requirements-doc`). Chinese or other non-conforming names are invalid.
112
+ Before creating OpenSpec artifacts, turn Step 1 clarification into a resolved brief containing the goal, non-goals, scope boundaries, key unknowns, and draft acceptance scenarios. Derive one kebab-case English change name that accurately represents that scope.
79
113
 
80
- When pausing, present:
81
- - **2-3 recommended kebab-case English names** derived from the confirmed clarification summary, each with a one-line description of the scope it implies
82
- - An explicit option for the user to **enter their own name**
83
- - A note that **if the user enters Chinese (or any non-kebab-case text), it will be converted into a compliant kebab-case English name**, and the converted result must be shown back to the user for confirmation before use
114
+ - **Continue directly when scope and naming are both unambiguous**. Do not pause merely to approve a summary or name; final review confirms the change name, scope, and artifacts together
115
+ - If the user supplied a name, normalize it to kebab-case and echo it in the progress update. Do not re-confirm when normalization preserves meaning
116
+ - Reuse a confirmed batch item's persisted summary and name. Re-clarify only when scope drift or missing manifest data is detected
117
+ - Use `comet/reference/decision-point.md` for one joint question only when mutually exclusive choices still change scope or the target change identity. Naming preference alone is not a blocking point
84
118
 
85
- The decision options must include:
86
- - Pick one of the recommended names
87
- - "Enter a custom name" — accept the user's input; if it is already valid kebab-case English, use it directly; if it is Chinese or otherwise non-conforming, convert it to compliant kebab-case English and show the converted name for confirmation before continuing
119
+ OpenSpec names must be kebab-case English using lowercase letters, digits, and single hyphens. When a collision exists but the target remains clear, derive a stable non-conflicting name and continue. Ask only when Comet cannot determine whether to reuse the existing change or create a new one.
88
120
 
89
- Must not run `openspec new change` or create `.comet.yaml` before the user confirms the final change name. If the chosen/converted name collides with an existing change, report the collision and ask the user to choose another name.
121
+ Do not run `openspec new change` or create proposal/design/tasks while the resolved brief or name remains ambiguous. Continue clarification or resolve the genuine user decision before Step 2.
90
122
 
91
123
  ### 2. Create Change Structure + Initialize State
92
124
 
@@ -94,35 +126,50 @@ Must not run `openspec new change` or create `.comet.yaml` before the user confi
94
126
 
95
127
  Full `/comet` workflow must not use the Skill tool to load the `openspec-propose` skill by default; only load it when the user explicitly requests generating the proposal and artifacts in one pass.
96
128
 
97
- After the skill loads, follow its guidance to create the change skeleton, but override its "STOP and wait for user direction" behavior when a confirmed clarification summary from Step 1b is already available in the conversation context.
129
+ After the skill loads, follow its guidance to create the change skeleton. When Step 1b has produced an unambiguous resolved brief, override its "STOP and wait for user direction" behavior to avoid a duplicate question.
130
+
131
+ Use the Step 1b resolved brief directly to populate artifact content. Fall back to the skill's question flow only when ambiguity remains that would change scope.
132
+
133
+ Immediately after creating the change skeleton, initialize recoverable state instead of waiting until every artifact is generated:
134
+
135
+ ```bash
136
+ comet state init <name> full
137
+ comet state select <name>
138
+ comet state check <name> open
139
+ ```
98
140
 
99
- If the user has already confirmed a clarification summary (Step 1b), use that summary directly to populate artifact content. If no clarification summary exists (edge case), fall back to the skill's default behavior of asking the user.
141
+ Stop if any command fails. Then run `openspec status --change "<name>" --json` once and perform compatibility preflight:
100
142
 
101
- After the change skeleton is created, generate `proposal`, `design`, and `tasks` one by one using the standard artifact loop:
143
+ - Resolved `changeRoot` must equal repository-local `openspec/changes/<name>`, and `planningHome` (when present) must remain inside the current repository
144
+ - `artifacts` must contain core ids `proposal`, `design`, and `tasks`; extra artifacts are allowed
145
+ - `applyRequires` must be a parseable list of artifact ids and every id must exist in `artifacts`
146
+ - Stop on missing fields, escaping paths, or missing core ids; never fall back to a guessed fixed template
102
147
 
103
- **Standard Artifact Loop** (for each `artifact-id`: `proposal` `design` `tasks`):
148
+ After preflight, generate the implementation-required artifacts from the OpenSpec schema and dependency graph:
104
149
 
105
- 1. Refresh status: `openspec status --change "<name>" --json`
106
- 2. Fetch artifact instructions:
150
+ **OpenSpec status-driven artifact loop**:
151
+
152
+ 1. Run `openspec status --change "<name>" --json` and parse the complete JSON.
153
+ 2. Exit when every item in `applyRequires` is `done`; record `isComplete` as diagnostic only and do not use it as a phase blocker.
154
+ 3. From unfinished `ready` artifacts, prioritize items that advance the `applyRequires` dependency closure and process them in CLI-returned order. Must not hard-code generation order or assume the schema contains only proposal/design/tasks.
155
+ 4. Fetch current instructions for each ready `<artifact-id>`:
107
156
 
108
157
  ```bash
109
- openspec instructions proposal --change "<name>" --json
110
- openspec instructions design --change "<name>" --json
111
- openspec instructions tasks --change "<name>" --json
158
+ openspec instructions <artifact-id> --change "<name>" --json
112
159
  ```
113
160
 
114
- 3. For the returned JSON instruction payload, you must:
161
+ 5. For the returned JSON instruction payload, you must:
115
162
  - Read every completed dependency artifact listed in `dependencies`
116
163
  - Use `template` as the artifact structure
117
164
  - Follow `instruction` guidance
118
- - Apply `context` and `rules` as constraints — **must not copy them into the artifact content**
119
- - Write to `resolvedOutputPath`
120
- - Verify the output file exists and is non-empty
121
- 4. After creating each artifact, re-run `openspec status --change "<name>" --json` to confirm status before continuing to the next artifact
165
+ - Apply `context` and `rules` as constraints — **must not copy them into artifact content**
166
+ - Write to `resolvedOutputPath`; for wildcard outputs, create each concrete file required by the instruction
167
+ - Verify the concrete output files returned by the CLI exist and are non-empty
168
+ 6. Re-run status after creating each artifact and revalidate `changeRoot`, core ids, and `applyRequires`. Do not regenerate items that become `done`; process newly `ready` items in the next loop.
122
169
 
123
- **Failure handling**: If `openspec instructions` fails, returns invalid JSON, reports unmet `dependencies`, or does not provide a usable `resolvedOutputPath`, must immediately stop artifact creation and report the OpenSpec error. Must not fall back to hard-coded artifact prose because that would silently bypass project rules.
170
+ **Blocking and failure handling**: if `applyRequires` is incomplete and no ready artifact can advance its dependency closure, report `missingDeps` for the relevant `blocked` artifacts and stop. Do not guess order or skip dependencies. Also stop if status/instructions fails, returns invalid JSON, escapes the repository, or provides no usable `resolvedOutputPath`. Must not fall back to hard-coded artifact prose.
124
171
 
125
- **Naming and scope guard**: Change name must be the kebab-case English name confirmed by the user in Step 1c must not auto-generate, infer, or use a non-kebab-case (e.g. Chinese) name. Change scope must match the user's description must not expand or narrow it independently.
172
+ **Naming and scope guard**: Use the kebab-case English name resolved in Step 1b; never use a non-kebab-case name. Change scope must match the resolved brief and user request; do not expand or narrow it independently.
126
173
 
127
174
  Confirm the following artifacts have been created:
128
175
 
@@ -135,43 +182,45 @@ openspec/changes/<name>/
135
182
  └── tasks.md # Task checklist (checkboxes)
136
183
  ```
137
184
 
138
- Create `.comet.yaml` state file:
139
-
140
- First locate scripts via `comet/reference/scripts.md`, then initialize state:
141
-
142
- ```bash
143
- node "$COMET_STATE" init <name> full
144
- ```
145
-
146
185
  ### 3. Entry State Verification
147
186
 
148
187
  Verify state machine has been correctly initialized:
149
188
 
150
189
  ```bash
151
- node "$COMET_STATE" check <name> open
190
+ comet state check <name> open
152
191
  ```
153
192
 
154
193
  Proceed to Step 4 after verification passes. The script outputs specific failure reasons when verification fails.
155
194
 
156
- **Idempotency**: All open phase operations can be safely re-executed. If `.comet.yaml` is already at `phase: open` and all three artifact files exist, skip completed steps and continue from the first missing step.
195
+ **Idempotent recovery algorithm**: all open phase operations can be safely re-executed. On recovery, process the status in this order:
196
+
197
+ 1. If state is missing, run `comet state init <name> full`; if malformed, stop and repair it instead of overwriting it. Then select the change and run `comet state check <name> open`.
198
+ 2. Run status and revalidate `changeRoot`, core ids, `applyRequires`, `artifacts`, and `missingDeps`.
199
+ 3. `done`: keep the artifact unchanged and do not regenerate it.
200
+ 4. `ready`: fetch its instructions, write the returned output, and immediately rerun status.
201
+ 5. `blocked`: follow `missingDeps` and first complete dependencies in the `applyRequires` closure; never generate a blocked artifact directly.
202
+ 6. Repeat until every item in `applyRequires` is `done`.
203
+
204
+ If the required dependency graph cannot advance, list the relevant blocked artifacts and `missingDeps`, then stop. Directory or fixed-file presence cannot replace the CLI decision; conversely, an optional artifact outside `applyRequires` must not block implementation solely because `isComplete` is false.
157
205
 
158
206
  ### 4. Content Completeness Check
159
207
 
160
- Confirm the three documents have complete content:
161
- - **proposal.md**: problem background, goals, scope, non-goals
162
- - **design.md**: high-level architecture decisions, approach selection, data flow
163
- - **tasks.md**: task list, each task has a clear description
208
+ Run status again. Confirm core ids exist, every item in `applyRequires` is `done`, and concrete files in `artifactPaths.<id>.existingOutputPaths` for required artifacts exist and are non-empty. If any condition fails, do not enter Step 5 or execute the phase guard.
164
209
 
165
- **File existence verification**: Confirm all three file paths exist and are non-empty. If any file is missing or empty, must not enter Step 5 or execute phase guard return to creation step to fill the gap.
210
+ Then check key artifact content: proposal covers problem, goals, scope, and non-goals; design covers high-level decisions and data flow; tasks contains clear work items. If the schema returns specs or other artifacts, check their content against their instructions as well; the fixed three documents must not hide an incomplete schema artifact.
166
211
 
167
212
  ### 5. User Review and Confirmation (Blocking Point)
168
213
 
169
- After the three documents are created and content completeness check passes, **must follow the `comet/reference/decision-point.md` protocol to pause and wait for user confirmation**. Must not execute phase guard or auto-transition before user confirmation.
214
+ After all OpenSpec artifacts are complete and the content check passes, **must follow the `comet/reference/decision-point.md` protocol to pause and wait for user confirmation**. Must not execute the phase guard or auto-transition before user confirmation.
215
+
216
+ The final review confirms the change name, scope, and artifact content together. Do not skip it because Step 1b resolved the brief, and do not add another routine summary/name confirmation before it.
170
217
 
171
218
  The user confirmation question must be presented as a single-select question with the following summary and options:
172
219
 
173
220
  **Summary content**:
221
+ - **Change name and resolved brief**: final name, goal, non-goals, scope boundaries, and key unknowns
174
222
  - **proposal.md**: problem background, goals, scope
223
+ - **specs and other schema artifacts**: capabilities, requirements, and key acceptance scenarios
175
224
  - **design.md**: high-level architecture decisions, approach selection
176
225
  - **tasks.md**: task count and key task descriptions
177
226
 
@@ -183,14 +232,14 @@ After user selects "Confirm", proceed to exit conditions. When user selects "Nee
183
232
 
184
233
  ## Exit Conditions
185
234
 
186
- - proposal.md, design.md, tasks.md all created with complete content
187
- - **User has confirmed** proposal, design, tasks content meets expectations
188
- - **Phase guard**: Run `node "$COMET_GUARD" <change-name> open --apply`; after all PASS, auto-transitions to next phase
235
+ - OpenSpec compatibility preflight passes, every `applyRequires` item is `done`, and required outputs are non-empty
236
+ - **User has confirmed** all OpenSpec artifact content meets expectations
237
+ - **Phase guard**: Run `comet guard <change-name> open --apply`; after all PASS, auto-transitions to next phase
189
238
 
190
239
  Must use `--apply` before exit, otherwise `.comet.yaml` remains at `phase: open` and the next phase entry check will fail.
191
240
 
192
241
  ```bash
193
- node "$COMET_GUARD" <change-name> open --apply
242
+ comet guard <change-name> open --apply
194
243
  ```
195
244
 
196
245
  Full workflow auto-transitions to `phase: design`; hotfix/tweak presets auto-transition to `phase: build`.
@@ -200,11 +249,11 @@ Full workflow auto-transitions to `phase: design`; hotfix/tweak presets auto-tra
200
249
  Follow `comet/reference/auto-transition.md`. Key command:
201
250
 
202
251
  ```bash
203
- node "$COMET_STATE" next <change-name>
252
+ comet state next <change-name>
204
253
  ```
205
254
 
206
255
  - `NEXT: auto` → invoke the skill pointed to by `SKILL` to enter the next phase
207
- - `NEXT: manual` → do not invoke the next skill; prompt user to run `/<SKILL>` manually
256
+ - `NEXT: manual` → do not invoke the next skill; return control with `HINT`, end the invocation, and do not create another confirmation point
208
257
  - `NEXT: done` → workflow is complete, no further action needed
209
258
 
210
259
  hotfix/tweak presets are controlled by their corresponding preset skill (phase goes directly to build); their `next` returns the corresponding preset skill.