@hassangameryt/freeflow 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (692) hide show
  1. package/.claude-plugin/marketplace.json +4 -4
  2. package/.claude-plugin/plugin.json +6 -2
  3. package/.codex-plugin/plugin.json +9 -9
  4. package/README.md +190 -127
  5. package/assets/feedback-loop.png +0 -0
  6. package/assets/freeflow_workflow_map.png +0 -0
  7. package/assets/workflow.png +0 -0
  8. package/{skills → capabilities}/output-router/SKILL.md +21 -17
  9. package/{skills → capabilities}/output-router/references/safety-policy.md +1 -1
  10. package/capabilities/output-router/references/setup.md +110 -0
  11. package/command-surface.json +62 -7
  12. package/hooks/freeflow-runtime-context.mjs +572 -154
  13. package/hooks/hooks.json +26 -1
  14. package/hooks/tests/check-runtime-context-hook.sh +375 -0
  15. package/package.json +24 -24
  16. package/pi-extension/dist/index.js +264 -80
  17. package/pi-extension/dist/integrations/mcp-config.js +29 -0
  18. package/pi-extension/dist/output-router/host-producer-identification.js +232 -0
  19. package/pi-extension/dist/output-router/native-safety-net.js +216 -0
  20. package/pi-extension/dist/output-router/observed-tool-routing.js +122 -0
  21. package/pi-extension/dist/output-router/renderers.js +1023 -0
  22. package/pi-extension/dist/output-router/router-tools.js +554 -0
  23. package/pi-extension/dist/output-router/schemas.js +383 -0
  24. package/pi-extension/dist/output-router/status.js +744 -0
  25. package/pi-extension/dist/output-router/utils.js +502 -0
  26. package/pi-extension/dist/runtime/runtime-context.js +798 -0
  27. package/pi-extension/dist/settings/settings-tui.js +314 -0
  28. package/pi-extension/dist/settings/settings-ui.js +1487 -0
  29. package/pi-extension/freeflow/index.js +1 -0
  30. package/pi-extension/src/index.ts +230 -56
  31. package/pi-extension/src/{host-producer-identification.ts → output-router/host-producer-identification.ts} +37 -7
  32. package/pi-extension/src/{native-safety-net.ts → output-router/native-safety-net.ts} +8 -12
  33. package/pi-extension/src/{observed-tool-routing.ts → output-router/observed-tool-routing.ts} +19 -11
  34. package/pi-extension/src/{renderers.ts → output-router/renderers.ts} +246 -55
  35. package/pi-extension/src/{router-tools.ts → output-router/router-tools.ts} +130 -29
  36. package/pi-extension/src/{schemas.ts → output-router/schemas.ts} +171 -43
  37. package/pi-extension/src/output-router/status.ts +791 -0
  38. package/pi-extension/src/{utils.ts → output-router/utils.ts} +80 -18
  39. package/pi-extension/src/runtime/runtime-context.ts +895 -0
  40. package/pi-extension/src/settings/settings-tui.ts +408 -0
  41. package/pi-extension/src/settings/settings-ui.ts +1730 -0
  42. package/pi-extension/tests/integration/pi-extension.test.js +4020 -0
  43. package/pi-extension/tests/output-router/observed-producers.test.js +132 -0
  44. package/pi-extension/tests/output-router/search.test.js +722 -0
  45. package/pi-extension/tests/settings/settings-tui.test.js +243 -0
  46. package/router/dist/benchmarks/benchmark-harness.d.ts +16 -13
  47. package/router/dist/benchmarks/benchmark-harness.js +62 -62
  48. package/router/dist/benchmarks/benchmarks.d.ts +80 -73
  49. package/router/dist/benchmarks/benchmarks.js +860 -761
  50. package/router/dist/benchmarks/codex-qa-benchmarks.d.ts +71 -67
  51. package/router/dist/benchmarks/codex-qa-benchmarks.js +429 -390
  52. package/router/dist/benchmarks/command-benchmarks.d.ts +107 -92
  53. package/router/dist/benchmarks/command-benchmarks.js +616 -571
  54. package/router/dist/benchmarks/context-mode-normalized-benchmarks.d.ts +64 -56
  55. package/router/dist/benchmarks/context-mode-normalized-benchmarks.js +679 -613
  56. package/router/dist/benchmarks/context-mode-real-deep-benchmark.d.ts +103 -95
  57. package/router/dist/benchmarks/context-mode-real-deep-benchmark.js +1188 -1103
  58. package/router/dist/benchmarks/context-saving-baseline.d.ts +47 -41
  59. package/router/dist/benchmarks/context-saving-baseline.js +405 -374
  60. package/router/dist/benchmarks/index-benchmarks.d.ts +74 -69
  61. package/router/dist/benchmarks/index-benchmarks.js +761 -661
  62. package/router/dist/benchmarks/storage-policy-benchmarks.d.ts +80 -69
  63. package/router/dist/benchmarks/storage-policy-benchmarks.js +549 -469
  64. package/router/dist/benchmarks/vault-index-benchmarks.d.ts +43 -34
  65. package/router/dist/benchmarks/vault-index-benchmarks.js +294 -229
  66. package/router/dist/config/config.d.ts +62 -55
  67. package/router/dist/config/config.js +552 -373
  68. package/router/dist/config/router-contract.d.ts +2 -2
  69. package/router/dist/config/router-contract.js +38 -38
  70. package/router/dist/config/schema.d.ts +19 -10
  71. package/router/dist/config/schema.js +586 -538
  72. package/router/dist/config/types.d.ts +413 -361
  73. package/router/dist/config/types.js +19 -8
  74. package/router/dist/evidence/bounded-evidence.d.ts +26 -26
  75. package/router/dist/evidence/bounded-evidence.js +265 -252
  76. package/router/dist/evidence/evidence-range-selector.d.ts +16 -14
  77. package/router/dist/evidence/evidence-range-selector.js +192 -191
  78. package/router/dist/evidence/evidence-search.d.ts +18 -16
  79. package/router/dist/evidence/evidence-search.js +401 -377
  80. package/router/dist/evidence/evidence.d.ts +17 -17
  81. package/router/dist/evidence/evidence.js +163 -159
  82. package/router/dist/evidence/failure-contracts.d.ts +25 -15
  83. package/router/dist/evidence/failure-contracts.js +132 -126
  84. package/router/dist/evidence/line-ranges.d.ts +17 -15
  85. package/router/dist/evidence/line-ranges.js +18 -18
  86. package/router/dist/experiments/local-index.d.ts +47 -41
  87. package/router/dist/experiments/local-index.js +477 -458
  88. package/router/dist/local/local-traversal.d.ts +18 -0
  89. package/router/dist/local/local-traversal.js +334 -0
  90. package/router/dist/processing/engine.d.ts +94 -70
  91. package/router/dist/processing/engine.js +462 -364
  92. package/router/dist/processing/reducers.d.ts +208 -194
  93. package/router/dist/processing/reducers.js +1434 -1256
  94. package/router/dist/processing/renderers.d.ts +24 -18
  95. package/router/dist/processing/renderers.js +161 -152
  96. package/router/dist/processing/scripts.d.ts +80 -67
  97. package/router/dist/processing/scripts.js +343 -290
  98. package/router/dist/repo/repo-traversal.d.ts +9 -9
  99. package/router/dist/repo/repo-traversal.js +205 -202
  100. package/router/dist/routing/observed-reducers.d.ts +25 -19
  101. package/router/dist/routing/observed-reducers.js +559 -545
  102. package/router/dist/routing/observed-routing.d.ts +23 -14
  103. package/router/dist/routing/observed-routing.js +319 -318
  104. package/router/dist/routing/parsers.d.ts +10 -10
  105. package/router/dist/routing/parsers.js +454 -406
  106. package/router/dist/routing/run-filters.d.ts +45 -41
  107. package/router/dist/routing/run-filters.js +215 -207
  108. package/router/dist/routing/run-reducers.d.ts +45 -26
  109. package/router/dist/routing/run-reducers.js +102 -90
  110. package/router/dist/sandbox/adapter-roots.d.ts +34 -26
  111. package/router/dist/sandbox/adapter-roots.js +43 -41
  112. package/router/dist/sandbox/eryx-python-adapter.d.ts +9 -5
  113. package/router/dist/sandbox/eryx-python-adapter.js +659 -588
  114. package/router/dist/sandbox/jq-wasm-adapter.d.ts +3 -3
  115. package/router/dist/sandbox/jq-wasm-adapter.js +319 -311
  116. package/router/dist/sandbox/quickjs-wasi-adapter.d.ts +9 -5
  117. package/router/dist/sandbox/quickjs-wasi-adapter.js +336 -333
  118. package/router/dist/sandbox/script-sandbox.d.ts +105 -84
  119. package/router/dist/sandbox/script-sandbox.js +294 -260
  120. package/router/dist/setup/script-transform-adapters.d.ts +23 -19
  121. package/router/dist/setup/script-transform-adapters.js +223 -204
  122. package/router/dist/tools/batch.d.ts +27 -16
  123. package/router/dist/tools/batch.js +493 -416
  124. package/router/dist/tools/run.d.ts +47 -35
  125. package/router/dist/tools/run.js +1172 -1087
  126. package/router/dist/tools/search.d.ts +53 -31
  127. package/router/dist/tools/search.js +1224 -1065
  128. package/router/dist/transform/engine.d.ts +121 -96
  129. package/router/dist/transform/engine.js +1853 -1736
  130. package/router/dist/vault/vault-index.d.ts +97 -78
  131. package/router/dist/vault/vault-index.js +347 -342
  132. package/router/dist/vault/vault.d.ts +109 -74
  133. package/router/dist/vault/vault.js +528 -533
  134. package/runtime/interaction-contract.md +16 -0
  135. package/skills/bypass/SKILL.md +52 -38
  136. package/skills/commit-work/SKILL.md +54 -46
  137. package/skills/commit-work/references/staging-decisions.md +46 -24
  138. package/skills/decision-gate/SKILL.md +48 -0
  139. package/skills/design-for-depth/SKILL.md +75 -59
  140. package/skills/design-for-depth/references/design-pressure-signals.md +56 -7
  141. package/skills/design-for-depth/references/interface-design-loop.md +156 -0
  142. package/skills/design-for-depth/references/software-design-philosophy.md +32 -3
  143. package/skills/diagnose-failure/SKILL.md +65 -60
  144. package/skills/diagnose-failure/references/diagnostic-loop-catalog.md +36 -0
  145. package/skills/diagnose-failure/references/flaky-and-performance.md +30 -18
  146. package/skills/discuss/SKILL.md +121 -0
  147. package/skills/discuss/references/checkpoints.md +66 -0
  148. package/skills/evaluate-skill/SKILL.md +62 -53
  149. package/skills/evaluate-skill/references/definition-schema.md +345 -0
  150. package/skills/evaluate-skill/references/evaluation-design.md +86 -0
  151. package/skills/evaluate-skill/references/execution-and-evidence.md +85 -0
  152. package/skills/evaluate-skill/references/review-and-revision.md +51 -0
  153. package/skills/evaluate-skill/scripts/lib/definitions.mjs +296 -0
  154. package/skills/evaluate-skill/scripts/lib/evidence.mjs +46 -0
  155. package/skills/evaluate-skill/scripts/lib/grade-comparison.mjs +60 -0
  156. package/skills/evaluate-skill/scripts/lib/grade-interaction.mjs +148 -0
  157. package/skills/evaluate-skill/scripts/lib/grade-support.mjs +126 -0
  158. package/skills/evaluate-skill/scripts/lib/grade-workspace.mjs +192 -0
  159. package/skills/evaluate-skill/scripts/lib/grade.mjs +56 -0
  160. package/skills/evaluate-skill/scripts/lib/pi.mjs +489 -0
  161. package/skills/evaluate-skill/scripts/lib/process.mjs +79 -0
  162. package/skills/evaluate-skill/scripts/lib/rpc.mjs +444 -0
  163. package/skills/evaluate-skill/scripts/lib/runner.mjs +346 -0
  164. package/skills/evaluate-skill/scripts/lib/sandbox.mjs +409 -0
  165. package/skills/evaluate-skill/scripts/lib/view.mjs +337 -0
  166. package/skills/evaluate-skill/scripts/pi-guard.mjs +164 -0
  167. package/skills/evaluate-skill/scripts/skill-eval.mjs +101 -0
  168. package/skills/execute-work/SKILL.md +99 -0
  169. package/skills/execute-work/references/code-practices.md +78 -0
  170. package/skills/execute-work/references/execution-loop.md +77 -0
  171. package/skills/finish-branch/SKILL.md +117 -0
  172. package/skills/finish-branch/references/integration-options.md +80 -0
  173. package/skills/handoff/SKILL.md +65 -39
  174. package/skills/handoff/references/templates.md +104 -27
  175. package/skills/launch-work/SKILL.md +116 -0
  176. package/skills/launch-work/references/launch-readiness.md +85 -0
  177. package/skills/migration-work/SKILL.md +115 -0
  178. package/skills/migration-work/references/migration-lifecycle.md +82 -0
  179. package/skills/mode-contract/SKILL.md +72 -52
  180. package/skills/release-work/SKILL.md +118 -0
  181. package/skills/release-work/references/release-evidence.md +78 -0
  182. package/skills/review-artifact/SKILL.md +77 -69
  183. package/skills/review-artifact/references/reviewer-prompt.md +133 -42
  184. package/skills/review-work/SKILL.md +82 -57
  185. package/skills/review-work/references/reviewer-prompt.md +97 -60
  186. package/skills/review-work/references/security-risk-lens.md +67 -0
  187. package/skills/setup-freeflow/SKILL.md +54 -68
  188. package/skills/setup-freeflow/references/activation-contract.md +84 -69
  189. package/skills/setup-freeflow/references/host-setup.md +50 -17
  190. package/skills/simplify-code/SKILL.md +80 -0
  191. package/skills/simplify-code/references/simplification-patterns.md +70 -0
  192. package/skills/tdd/SKILL.md +96 -0
  193. package/skills/tdd/references/test-design.md +96 -0
  194. package/skills/track-work/SKILL.md +84 -0
  195. package/skills/track-work/references/working-record-schema.md +378 -0
  196. package/skills/verify-work/SKILL.md +59 -38
  197. package/skills/verify-work/references/browser-runtime-evidence.md +55 -0
  198. package/skills/verify-work/references/integration-evidence.md +51 -0
  199. package/skills/verify-work/references/performance-evidence.md +57 -0
  200. package/skills/workflow/SKILL.md +53 -59
  201. package/skills/workflow/references/domain-skill-composition.md +57 -0
  202. package/skills/workflow/references/workflow-loop.md +92 -0
  203. package/skills/write-plan/SKILL.md +85 -58
  204. package/skills/write-plan/references/plan-shapes.md +132 -46
  205. package/skills/write-skill/SKILL.md +82 -51
  206. package/skills/write-skill/references/activation-boundaries.md +62 -0
  207. package/skills/write-skill/references/agent-first-instructions.md +87 -0
  208. package/skills/write-skill/references/progressive-disclosure.md +63 -0
  209. package/skills/write-skill/scripts/lib/skill-author.mjs +653 -0
  210. package/skills/write-skill/scripts/skill-author.mjs +104 -0
  211. package/skills/write-spec/SKILL.md +65 -54
  212. package/skills/write-spec/references/artifact-standards.md +60 -41
  213. package/skills/write-spec/references/decision-records.md +77 -0
  214. package/skills/write-spec/references/spec-shapes.md +119 -60
  215. package/evals/README.md +0 -32
  216. package/evals/fixtures/copy-conventions.md +0 -31
  217. package/evals/fixtures/output-router/README.md +0 -12
  218. package/evals/fixtures/output-router/failed-command-output.txt +0 -9
  219. package/evals/fixtures/output-router/large-router-manual.md +0 -63
  220. package/evals/fixtures/output-router/native-large-output.txt +0 -40
  221. package/evals/fixtures/output-router/noisy-test-output.txt +0 -30
  222. package/evals/fixtures/output-router/verification-output.txt +0 -8
  223. package/evals/fixtures/tiny-artifact-review-app/README.md +0 -5
  224. package/evals/fixtures/tiny-artifact-review-app/docs/billing-policy.md +0 -10
  225. package/evals/fixtures/tiny-artifact-review-app/docs/specs/billing-webhook-api-draft.md +0 -21
  226. package/evals/fixtures/tiny-artifact-review-app/docs/specs/failed-payment-grace-period.md +0 -24
  227. package/evals/fixtures/tiny-artifact-review-app/docs/specs/immediate-downgrade.md +0 -17
  228. package/evals/fixtures/tiny-artifact-review-app/tests/billing.test.md +0 -8
  229. package/evals/fixtures/tiny-commit-ambiguous-app/.freeflow-eval-setup.sh +0 -15
  230. package/evals/fixtures/tiny-commit-ambiguous-app/README.md +0 -3
  231. package/evals/fixtures/tiny-commit-ambiguous-app/src/billing.ts +0 -5
  232. package/evals/fixtures/tiny-commit-ambiguous-app/src/settings-page.ts +0 -1
  233. package/evals/fixtures/tiny-commit-ambiguous-app/tests/settings.test.md +0 -5
  234. package/evals/fixtures/tiny-commit-app/.freeflow-eval-setup.sh +0 -14
  235. package/evals/fixtures/tiny-commit-app/README.md +0 -3
  236. package/evals/fixtures/tiny-commit-app/docs/specs/settings-title.md +0 -5
  237. package/evals/fixtures/tiny-commit-app/src/billing.ts +0 -5
  238. package/evals/fixtures/tiny-commit-app/src/settings-page.ts +0 -1
  239. package/evals/fixtures/tiny-commit-app/tests/settings.test.md +0 -5
  240. package/evals/fixtures/tiny-commit-everything-app/.freeflow-eval-setup.sh +0 -15
  241. package/evals/fixtures/tiny-commit-everything-app/README.md +0 -3
  242. package/evals/fixtures/tiny-commit-everything-app/docs/specs/settings-title.md +0 -5
  243. package/evals/fixtures/tiny-commit-everything-app/src/billing.ts +0 -5
  244. package/evals/fixtures/tiny-commit-everything-app/src/settings-page.ts +0 -1
  245. package/evals/fixtures/tiny-commit-everything-app/tests/settings.test.md +0 -5
  246. package/evals/fixtures/tiny-commit-staged-app/.freeflow-eval-setup.sh +0 -15
  247. package/evals/fixtures/tiny-commit-staged-app/README.md +0 -3
  248. package/evals/fixtures/tiny-commit-staged-app/docs/specs/settings-title.md +0 -5
  249. package/evals/fixtures/tiny-commit-staged-app/src/billing.ts +0 -5
  250. package/evals/fixtures/tiny-commit-staged-app/src/settings-page.ts +0 -1
  251. package/evals/fixtures/tiny-commit-staged-app/tests/settings.test.md +0 -5
  252. package/evals/fixtures/tiny-commit-staged-mixed-app/.freeflow-eval-setup.sh +0 -15
  253. package/evals/fixtures/tiny-commit-staged-mixed-app/README.md +0 -3
  254. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/billing-policy.md +0 -3
  255. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/release-notes.md +0 -3
  256. package/evals/fixtures/tiny-commit-staged-mixed-app/src/billing.ts +0 -5
  257. package/evals/fixtures/tiny-dashboard-stale-app/README.md +0 -11
  258. package/evals/fixtures/tiny-dashboard-stale-app/src/api-client.ts +0 -13
  259. package/evals/fixtures/tiny-dashboard-stale-app/src/dashboard-data.ts +0 -19
  260. package/evals/fixtures/tiny-dashboard-stale-app/tests/dashboard-data.test.md +0 -9
  261. package/evals/fixtures/tiny-decision-destination-app/README.md +0 -6
  262. package/evals/fixtures/tiny-decision-destination-app/src/risk.ts +0 -8
  263. package/evals/fixtures/tiny-decision-destination-app/tests/risk.test.md +0 -4
  264. package/evals/fixtures/tiny-design-depth-app/README.md +0 -15
  265. package/evals/fixtures/tiny-design-depth-app/src/routes.ts +0 -25
  266. package/evals/fixtures/tiny-design-depth-app/src/sendEmail.ts +0 -3
  267. package/evals/fixtures/tiny-design-depth-app/src/sendSms.ts +0 -3
  268. package/evals/fixtures/tiny-design-depth-app/src/telemetry.ts +0 -3
  269. package/evals/fixtures/tiny-design-depth-app/tests/routes.test.md +0 -12
  270. package/evals/fixtures/tiny-eval-method-app/README.md +0 -8
  271. package/evals/fixtures/tiny-eval-method-app/evals/reports/review-pr-eval-result.md +0 -5
  272. package/evals/fixtures/tiny-eval-method-app/evals/reports/triage-question-failure.md +0 -17
  273. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.diff +0 -11
  274. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.md +0 -3
  275. package/evals/fixtures/tiny-eval-method-app/skills/review-pr/SKILL.md +0 -10
  276. package/evals/fixtures/tiny-eval-method-app/skills/triage-question/SKILL.md +0 -10
  277. package/evals/fixtures/tiny-execute-plan-app/README.md +0 -5
  278. package/evals/fixtures/tiny-execute-plan-app/checks/settings-and-billing-check.sh +0 -5
  279. package/evals/fixtures/tiny-execute-plan-app/docs/billing-policy.md +0 -10
  280. package/evals/fixtures/tiny-execute-plan-app/docs/plans/immediate-downgrade-plan.md +0 -11
  281. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-plan.md +0 -12
  282. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-with-bad-check.md +0 -15
  283. package/evals/fixtures/tiny-execute-plan-app/docs/plans/unverified-billing-plan.md +0 -9
  284. package/evals/fixtures/tiny-execute-plan-app/docs/specs/failed-payment-grace-period.md +0 -9
  285. package/evals/fixtures/tiny-execute-plan-app/docs/specs/settings-title.md +0 -9
  286. package/evals/fixtures/tiny-execute-plan-app/src/billing.ts +0 -7
  287. package/evals/fixtures/tiny-execute-plan-app/src/settings-page.ts +0 -1
  288. package/evals/fixtures/tiny-execute-plan-app/tests/billing.test.md +0 -7
  289. package/evals/fixtures/tiny-execute-plan-app/tests/settings.test.md +0 -5
  290. package/evals/fixtures/tiny-execute-review-loop-app/README.md +0 -5
  291. package/evals/fixtures/tiny-execute-review-loop-app/docs/billing-policy.md +0 -9
  292. package/evals/fixtures/tiny-execute-review-loop-app/docs/plans/settings-title-plan.md +0 -12
  293. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/previous-agent-summary.md +0 -15
  294. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/review-pass-1-non-pass.md +0 -11
  295. package/evals/fixtures/tiny-execute-review-loop-app/docs/specs/settings-title.md +0 -9
  296. package/evals/fixtures/tiny-execute-review-loop-app/src/billing.ts +0 -7
  297. package/evals/fixtures/tiny-execute-review-loop-app/src/settings-page.ts +0 -1
  298. package/evals/fixtures/tiny-execute-review-loop-app/tests/billing.test.md +0 -7
  299. package/evals/fixtures/tiny-execute-review-loop-app/tests/settings.test.md +0 -5
  300. package/evals/fixtures/tiny-flaky-session-app/README.md +0 -10
  301. package/evals/fixtures/tiny-flaky-session-app/src/session-timeout.ts +0 -13
  302. package/evals/fixtures/tiny-flaky-session-app/tests/session-timeout.test.md +0 -8
  303. package/evals/fixtures/tiny-handoff-app/README.md +0 -5
  304. package/evals/fixtures/tiny-handoff-app/docs/billing-policy.md +0 -10
  305. package/evals/fixtures/tiny-handoff-app/docs/handoffs/old-agent-handoff.md +0 -8
  306. package/evals/fixtures/tiny-handoff-app/docs/plans/billing-retry-plan.md +0 -11
  307. package/evals/fixtures/tiny-handoff-app/src/billing.ts +0 -7
  308. package/evals/fixtures/tiny-handoff-report-app/README.md +0 -3
  309. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.diff +0 -1
  310. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.md +0 -1
  311. package/evals/fixtures/tiny-handoff-report-app/evals/scores/hof-003-score.txt +0 -4
  312. package/evals/fixtures/tiny-plan-app/README.md +0 -5
  313. package/evals/fixtures/tiny-plan-app/docs/billing-policy.md +0 -10
  314. package/evals/fixtures/tiny-plan-app/docs/specs/failed-payment-grace-period.md +0 -25
  315. package/evals/fixtures/tiny-plan-app/src/billing.ts +0 -17
  316. package/evals/fixtures/tiny-plan-app/src/settings-page.ts +0 -1
  317. package/evals/fixtures/tiny-plan-app/tests/billing.test.md +0 -8
  318. package/evals/fixtures/tiny-plan-app/tests/settings.test.md +0 -5
  319. package/evals/fixtures/tiny-plan-strict-api-app/README.md +0 -3
  320. package/evals/fixtures/tiny-plan-strict-api-app/docs/billing-policy.md +0 -5
  321. package/evals/fixtures/tiny-plan-strict-api-app/docs/specs/billing-webhook-api.md +0 -25
  322. package/evals/fixtures/tiny-plan-strict-api-app/src/billing.ts +0 -5
  323. package/evals/fixtures/tiny-plan-strict-api-app/src/webhooks.ts +0 -3
  324. package/evals/fixtures/tiny-plan-strict-api-app/tests/billing-webhook.test.md +0 -8
  325. package/evals/fixtures/tiny-post-setup-invalid-config-app/.freeflow/config.json +0 -3
  326. package/evals/fixtures/tiny-post-setup-invalid-config-app/AGENTS.md +0 -15
  327. package/evals/fixtures/tiny-post-setup-invalid-config-app/README.md +0 -16
  328. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/auth-review.md +0 -15
  329. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/billing-policy.md +0 -10
  330. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/copy-conventions.md +0 -15
  331. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/auth-middleware.ts +0 -12
  332. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/billing.ts +0 -20
  333. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/login-copy.ts +0 -1
  334. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/settings-page.ts +0 -1
  335. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/auth-middleware.test.md +0 -8
  336. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/billing.test.md +0 -8
  337. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/copy.test.md +0 -7
  338. package/evals/fixtures/tiny-post-setup-missing-config-app/AGENTS.md +0 -15
  339. package/evals/fixtures/tiny-post-setup-missing-config-app/README.md +0 -16
  340. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/auth-review.md +0 -15
  341. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/billing-policy.md +0 -10
  342. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/copy-conventions.md +0 -15
  343. package/evals/fixtures/tiny-post-setup-missing-config-app/src/auth-middleware.ts +0 -12
  344. package/evals/fixtures/tiny-post-setup-missing-config-app/src/billing.ts +0 -20
  345. package/evals/fixtures/tiny-post-setup-missing-config-app/src/login-copy.ts +0 -1
  346. package/evals/fixtures/tiny-post-setup-missing-config-app/src/settings-page.ts +0 -1
  347. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/auth-middleware.test.md +0 -8
  348. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/billing.test.md +0 -8
  349. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/copy.test.md +0 -7
  350. package/evals/fixtures/tiny-post-setup-source-conflict-app/.freeflow/config.json +0 -3
  351. package/evals/fixtures/tiny-post-setup-source-conflict-app/AGENTS.md +0 -15
  352. package/evals/fixtures/tiny-post-setup-source-conflict-app/README.md +0 -16
  353. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/auth-review.md +0 -15
  354. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/billing-policy.md +0 -10
  355. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/copy-conventions.md +0 -15
  356. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/auth-middleware.ts +0 -12
  357. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/billing.ts +0 -20
  358. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/login-copy.ts +0 -1
  359. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/settings-page.ts +0 -1
  360. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/auth-middleware.test.md +0 -8
  361. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/billing.test.md +0 -8
  362. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/copy.test.md +0 -7
  363. package/evals/fixtures/tiny-report-performance-app/README.md +0 -11
  364. package/evals/fixtures/tiny-report-performance-app/docs/performance.md +0 -10
  365. package/evals/fixtures/tiny-report-performance-app/src/report-summary.ts +0 -29
  366. package/evals/fixtures/tiny-report-performance-app/tests/report-performance.test.md +0 -8
  367. package/evals/fixtures/tiny-review-request-app/README.md +0 -3
  368. package/evals/fixtures/tiny-review-request-app/docs/billing-policy.md +0 -5
  369. package/evals/fixtures/tiny-review-request-app/docs/plans/billing-webhook-api.md +0 -19
  370. package/evals/fixtures/tiny-review-request-app/docs/review/previous-agent-summary.md +0 -20
  371. package/evals/fixtures/tiny-review-request-app/docs/specs/billing-webhook-api.md +0 -25
  372. package/evals/fixtures/tiny-review-request-app/src/billing.ts +0 -11
  373. package/evals/fixtures/tiny-review-request-app/src/webhooks.ts +0 -19
  374. package/evals/fixtures/tiny-review-request-app/tests/billing-webhook.test.md +0 -8
  375. package/evals/fixtures/tiny-review-work-app/README.md +0 -5
  376. package/evals/fixtures/tiny-review-work-app/docs/billing-policy.md +0 -9
  377. package/evals/fixtures/tiny-review-work-app/docs/plans/settings-title-plan.md +0 -12
  378. package/evals/fixtures/tiny-review-work-app/docs/review/previous-agent-summary.md +0 -15
  379. package/evals/fixtures/tiny-review-work-app/docs/specs/settings-title.md +0 -9
  380. package/evals/fixtures/tiny-review-work-app/src/billing.ts +0 -7
  381. package/evals/fixtures/tiny-review-work-app/src/settings-page.ts +0 -1
  382. package/evals/fixtures/tiny-review-work-app/tests/billing.test.md +0 -7
  383. package/evals/fixtures/tiny-review-work-app/tests/settings.test.md +0 -5
  384. package/evals/fixtures/tiny-saas-app/README.md +0 -16
  385. package/evals/fixtures/tiny-saas-app/docs/auth-review.md +0 -15
  386. package/evals/fixtures/tiny-saas-app/docs/billing-policy.md +0 -10
  387. package/evals/fixtures/tiny-saas-app/docs/copy-conventions.md +0 -15
  388. package/evals/fixtures/tiny-saas-app/docs/specs/settings-title.md +0 -13
  389. package/evals/fixtures/tiny-saas-app/src/auth-middleware.ts +0 -12
  390. package/evals/fixtures/tiny-saas-app/src/billing.ts +0 -20
  391. package/evals/fixtures/tiny-saas-app/src/login-copy.ts +0 -1
  392. package/evals/fixtures/tiny-saas-app/src/settings-page.ts +0 -1
  393. package/evals/fixtures/tiny-saas-app/tests/auth-middleware.test.md +0 -8
  394. package/evals/fixtures/tiny-saas-app/tests/billing.test.md +0 -8
  395. package/evals/fixtures/tiny-saas-app/tests/copy.test.md +0 -7
  396. package/evals/fixtures/tiny-setup-agents-app/AGENTS.md +0 -6
  397. package/evals/fixtures/tiny-setup-both-app/AGENTS.md +0 -5
  398. package/evals/fixtures/tiny-setup-both-app/CLAUDE.md +0 -5
  399. package/evals/fixtures/tiny-setup-claude-app/CLAUDE.md +0 -6
  400. package/evals/fixtures/tiny-setup-codex-rules-app/.codex/rules/shell.rules +0 -5
  401. package/evals/fixtures/tiny-setup-codex-rules-app/AGENTS.md +0 -6
  402. package/evals/fixtures/tiny-setup-conflict-app/AGENTS.md +0 -6
  403. package/evals/fixtures/tiny-setup-existing-block-app/AGENTS.md +0 -14
  404. package/evals/fixtures/tiny-setup-four-rules-app/CLAUDE.md +0 -6
  405. package/evals/fixtures/tiny-skill-dev-app/README.md +0 -10
  406. package/evals/fixtures/tiny-skill-dev-app/evals/prompts/review-pr-001.txt +0 -8
  407. package/evals/fixtures/tiny-skill-dev-app/evals/reports/review-pr-failure.md +0 -17
  408. package/evals/fixtures/tiny-skill-dev-app/skills/review-pr/SKILL.md +0 -13
  409. package/evals/fixtures/tiny-team-app/README.md +0 -5
  410. package/evals/fixtures/tiny-team-app/docs/product.md +0 -6
  411. package/evals/fixtures/tiny-team-app/src/teams.ts +0 -9
  412. package/evals/fixtures/tiny-team-app/src/users.ts +0 -5
  413. package/evals/fixtures/tiny-team-app/tests/teams.test.md +0 -7
  414. package/evals/fixtures/tiny-verify-app/README.md +0 -5
  415. package/evals/fixtures/tiny-verify-app/src/settings-nav.ts +0 -1
  416. package/evals/fixtures/tiny-verify-app/src/settings-page.ts +0 -1
  417. package/evals/fixtures/tiny-verify-app/tests/settings.test.md +0 -7
  418. package/evals/prompts/aon-001.txt +0 -2
  419. package/evals/prompts/byp-001.txt +0 -2
  420. package/evals/prompts/byp-002.txt +0 -2
  421. package/evals/prompts/cap-001.txt +0 -3
  422. package/evals/prompts/cap-002.txt +0 -3
  423. package/evals/prompts/cmd-001.txt +0 -3
  424. package/evals/prompts/cmd-002.txt +0 -3
  425. package/evals/prompts/cmd-003.txt +0 -6
  426. package/evals/prompts/cmd-004.txt +0 -5
  427. package/evals/prompts/cmd-005.txt +0 -5
  428. package/evals/prompts/cmd-006.txt +0 -5
  429. package/evals/prompts/cmd-007.txt +0 -5
  430. package/evals/prompts/cmd-008.txt +0 -5
  431. package/evals/prompts/cmd-009.txt +0 -8
  432. package/evals/prompts/cmd-010.txt +0 -5
  433. package/evals/prompts/cmd-011.txt +0 -5
  434. package/evals/prompts/cmd-012.txt +0 -5
  435. package/evals/prompts/cmd-013.txt +0 -5
  436. package/evals/prompts/cmd-014.txt +0 -7
  437. package/evals/prompts/cmd-015.txt +0 -7
  438. package/evals/prompts/cmd-016.txt +0 -5
  439. package/evals/prompts/cmt-001.txt +0 -1
  440. package/evals/prompts/cmt-002.txt +0 -1
  441. package/evals/prompts/cmt-003.txt +0 -1
  442. package/evals/prompts/cmt-004.txt +0 -1
  443. package/evals/prompts/cmt-005.txt +0 -1
  444. package/evals/prompts/dfd-001.txt +0 -3
  445. package/evals/prompts/dia-001.txt +0 -3
  446. package/evals/prompts/dia-002.txt +0 -3
  447. package/evals/prompts/dia-003.txt +0 -3
  448. package/evals/prompts/dis-001.txt +0 -5
  449. package/evals/prompts/dis-002.txt +0 -5
  450. package/evals/prompts/esk-001.txt +0 -5
  451. package/evals/prompts/esk-002.txt +0 -5
  452. package/evals/prompts/esk-003.txt +0 -5
  453. package/evals/prompts/grc-001.txt +0 -3
  454. package/evals/prompts/hof-001.txt +0 -3
  455. package/evals/prompts/hof-002.txt +0 -1
  456. package/evals/prompts/hof-003.txt +0 -1
  457. package/evals/prompts/hof-004.txt +0 -1
  458. package/evals/prompts/hof-005.txt +0 -1
  459. package/evals/prompts/ivg-001.txt +0 -1
  460. package/evals/prompts/ivg-002.txt +0 -11
  461. package/evals/prompts/ivg-003.txt +0 -1
  462. package/evals/prompts/ivg-004.txt +0 -1
  463. package/evals/prompts/mode-001.txt +0 -3
  464. package/evals/prompts/mode-002.txt +0 -1
  465. package/evals/prompts/mode-003.txt +0 -3
  466. package/evals/prompts/mode-004.txt +0 -3
  467. package/evals/prompts/mode-005.txt +0 -3
  468. package/evals/prompts/mode-006.txt +0 -3
  469. package/evals/prompts/mode-007.txt +0 -3
  470. package/evals/prompts/otr-001.txt +0 -3
  471. package/evals/prompts/otr-002.txt +0 -3
  472. package/evals/prompts/rar-001.txt +0 -5
  473. package/evals/prompts/rar-002.txt +0 -3
  474. package/evals/prompts/rar-003.txt +0 -6
  475. package/evals/prompts/rar-004.txt +0 -15
  476. package/evals/prompts/rar-005.txt +0 -10
  477. package/evals/prompts/rbr-001.txt +0 -3
  478. package/evals/prompts/rbr-002.txt +0 -3
  479. package/evals/prompts/rbr-003.txt +0 -7
  480. package/evals/prompts/rev-001.txt +0 -3
  481. package/evals/prompts/rev-002.txt +0 -5
  482. package/evals/prompts/rev-003.txt +0 -6
  483. package/evals/prompts/rev-004.txt +0 -1
  484. package/evals/prompts/rev-005.txt +0 -15
  485. package/evals/prompts/rev-006.txt +0 -5
  486. package/evals/prompts/stp-001.txt +0 -1
  487. package/evals/prompts/stp-002.txt +0 -1
  488. package/evals/prompts/stp-003.txt +0 -1
  489. package/evals/prompts/stp-004.txt +0 -1
  490. package/evals/prompts/stp-005.txt +0 -1
  491. package/evals/prompts/stp-006.txt +0 -1
  492. package/evals/prompts/stp-007.txt +0 -1
  493. package/evals/prompts/stp-008.txt +0 -1
  494. package/evals/prompts/stp-009.txt +0 -1
  495. package/evals/prompts/stp-010.txt +0 -1
  496. package/evals/prompts/stp-011.txt +0 -1
  497. package/evals/prompts/vfy-003.txt +0 -1
  498. package/evals/prompts/wpl-001.txt +0 -3
  499. package/evals/prompts/wpl-002.txt +0 -5
  500. package/evals/prompts/wpl-003.txt +0 -3
  501. package/evals/prompts/wpl-004.txt +0 -5
  502. package/evals/prompts/wpl-005.txt +0 -1
  503. package/evals/prompts/wrc-001.txt +0 -10
  504. package/evals/prompts/wrc-002.txt +0 -3
  505. package/evals/prompts/wsk-001.txt +0 -6
  506. package/evals/prompts/wsp-001.txt +0 -5
  507. package/evals/prompts/wsp-002.txt +0 -15
  508. package/evals/prompts/wsp-003.txt +0 -3
  509. package/evals/prompts/wsp-004.txt +0 -15
  510. package/evals/prompts/wsp-005.txt +0 -5
  511. package/evals/prompts/wsp-006.txt +0 -12
  512. package/evals/prompts/wsp-007.txt +0 -9
  513. package/evals/prompts/wsp-008.txt +0 -13
  514. package/evals/prompts/xpl-001.txt +0 -1
  515. package/evals/prompts/xpl-002.txt +0 -3
  516. package/evals/prompts/xpl-003.txt +0 -3
  517. package/evals/prompts/xpl-004.txt +0 -3
  518. package/evals/prompts/xpl-005.txt +0 -5
  519. package/evals/prompts/xpl-006.txt +0 -3
  520. package/evals/registries/adversarial-evals.json +0 -39
  521. package/evals/registries/fixture-evals.json +0 -2513
  522. package/evals/registries/handoff-evals.json +0 -74
  523. package/evals/registries/legacy-evals.json +0 -147
  524. package/evals/registries/skill-evidence.json +0 -193
  525. package/evals/registries/verify-work-evals.json +0 -46
  526. package/evals/reports/acceptance/v0.1-acceptance-report.md +0 -130
  527. package/evals/reports/by-command-surface/command-surface-1-report.md +0 -45
  528. package/evals/reports/by-command-surface/command-surface-10-report.md +0 -76
  529. package/evals/reports/by-command-surface/command-surface-11-report.md +0 -77
  530. package/evals/reports/by-command-surface/command-surface-12-report.md +0 -91
  531. package/evals/reports/by-command-surface/command-surface-13-report.md +0 -64
  532. package/evals/reports/by-command-surface/command-surface-14-report.md +0 -66
  533. package/evals/reports/by-command-surface/command-surface-15-report.md +0 -124
  534. package/evals/reports/by-command-surface/command-surface-16-report.md +0 -39
  535. package/evals/reports/by-command-surface/command-surface-2-report.md +0 -80
  536. package/evals/reports/by-command-surface/command-surface-3-report.md +0 -86
  537. package/evals/reports/by-command-surface/command-surface-4-report.md +0 -76
  538. package/evals/reports/by-command-surface/command-surface-5-report.md +0 -80
  539. package/evals/reports/by-command-surface/command-surface-6-report.md +0 -74
  540. package/evals/reports/by-command-surface/command-surface-7-report.md +0 -84
  541. package/evals/reports/by-command-surface/command-surface-8-report.md +0 -79
  542. package/evals/reports/by-command-surface/command-surface-9-report.md +0 -79
  543. package/evals/reports/by-command-surface/command-surface-matrix.md +0 -91
  544. package/evals/reports/by-skill/bypass-1-report.md +0 -124
  545. package/evals/reports/by-skill/capture-decisions-1-report.md +0 -99
  546. package/evals/reports/by-skill/capture-decisions-2-report.md +0 -102
  547. package/evals/reports/by-skill/commit-work-1-report.md +0 -165
  548. package/evals/reports/by-skill/commit-work-2-report.md +0 -109
  549. package/evals/reports/by-skill/design-for-depth-1-report.md +0 -95
  550. package/evals/reports/by-skill/diagnose-failure-1-report.md +0 -91
  551. package/evals/reports/by-skill/diagnose-failure-2-report.md +0 -94
  552. package/evals/reports/by-skill/discover-1-report.md +0 -118
  553. package/evals/reports/by-skill/evaluate-skill-2-report.md +0 -71
  554. package/evals/reports/by-skill/execute-plan-1-report.md +0 -103
  555. package/evals/reports/by-skill/execute-plan-2-report.md +0 -112
  556. package/evals/reports/by-skill/execute-plan-3-report.md +0 -87
  557. package/evals/reports/by-skill/grill-context-1-report.md +0 -103
  558. package/evals/reports/by-skill/handoff-1-report.md +0 -123
  559. package/evals/reports/by-skill/handoff-2-report.md +0 -105
  560. package/evals/reports/by-skill/handoff-3-report.md +0 -100
  561. package/evals/reports/by-skill/handoff-7-report.md +0 -77
  562. package/evals/reports/by-skill/interview-gate-1-report.md +0 -63
  563. package/evals/reports/by-skill/interview-gate-2-report.md +0 -105
  564. package/evals/reports/by-skill/mode-contract-1-report.md +0 -74
  565. package/evals/reports/by-skill/mode-contract-2-report.md +0 -87
  566. package/evals/reports/by-skill/mode-contract-3-report.md +0 -65
  567. package/evals/reports/by-skill/mode-contract-4-report.md +0 -71
  568. package/evals/reports/by-skill/mode-contract-5-report.md +0 -70
  569. package/evals/reports/by-skill/output-router-1-report.md +0 -76
  570. package/evals/reports/by-skill/output-router-2-report.md +0 -65
  571. package/evals/reports/by-skill/research-1-report.md +0 -144
  572. package/evals/reports/by-skill/research-brief-1-report.md +0 -94
  573. package/evals/reports/by-skill/research-brief-2-report.md +0 -75
  574. package/evals/reports/by-skill/review-artifact-1-report.md +0 -91
  575. package/evals/reports/by-skill/review-artifact-2-report.md +0 -79
  576. package/evals/reports/by-skill/review-artifact-3-report.md +0 -120
  577. package/evals/reports/by-skill/review-artifact-4-report.md +0 -89
  578. package/evals/reports/by-skill/review-artifact-5-report.md +0 -66
  579. package/evals/reports/by-skill/review-work-1-report.md +0 -85
  580. package/evals/reports/by-skill/review-work-2-report.md +0 -87
  581. package/evals/reports/by-skill/review-work-3-report.md +0 -104
  582. package/evals/reports/by-skill/review-work-4-report.md +0 -108
  583. package/evals/reports/by-skill/review-work-5-report.md +0 -73
  584. package/evals/reports/by-skill/review-work-6-report.md +0 -66
  585. package/evals/reports/by-skill/setup-freeflow-1-report.md +0 -199
  586. package/evals/reports/by-skill/setup-freeflow-2-report.md +0 -105
  587. package/evals/reports/by-skill/setup-freeflow-3-report.md +0 -105
  588. package/evals/reports/by-skill/setup-freeflow-4-report.md +0 -88
  589. package/evals/reports/by-skill/setup-freeflow-5-report.md +0 -115
  590. package/evals/reports/by-skill/skill-authoring-1-report.md +0 -81
  591. package/evals/reports/by-skill/verify-work-1-report.md +0 -111
  592. package/evals/reports/by-skill/verify-work-2-report.md +0 -120
  593. package/evals/reports/by-skill/workflow-1-report.md +0 -65
  594. package/evals/reports/by-skill/write-plan-1-report.md +0 -123
  595. package/evals/reports/by-skill/write-plan-2-report.md +0 -98
  596. package/evals/reports/by-skill/write-plan-3-report.md +0 -110
  597. package/evals/reports/by-skill/write-spec-1-report.md +0 -88
  598. package/evals/reports/by-skill/write-spec-2-report.md +0 -89
  599. package/evals/reports/by-skill/write-spec-3-report.md +0 -90
  600. package/evals/reports/by-skill/write-spec-4-report.md +0 -167
  601. package/evals/reports/harness/claude-harness-1-report.md +0 -103
  602. package/evals/reports/harness/eval-harness-1-report.md +0 -91
  603. package/evals/reports/iterations/iteration-1-report.md +0 -71
  604. package/evals/reports/iterations/iteration-2-report.md +0 -78
  605. package/evals/reports/iterations/iteration-3-report.md +0 -118
  606. package/evals/reports/iterations/iteration-4-report.md +0 -131
  607. package/evals/reports/iterations/iteration-5-report.md +0 -149
  608. package/evals/reports/iterations/iteration-6-report.md +0 -136
  609. package/evals/reports/runtime/always-on-runtime-1-report.md +0 -107
  610. package/evals/reports/runtime/context-mode-normalized-benchmark-1-report.md +0 -52
  611. package/evals/reports/runtime/context-mode-real-deep-baseline-1-report.md +0 -410
  612. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-report.md +0 -203
  613. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-review.md +0 -51
  614. package/evals/reports/runtime/context-saving-native-tools-baseline-1-report.md +0 -99
  615. package/evals/reports/runtime/eryx-python-compatibility-reprobe-1-report.md +0 -166
  616. package/evals/reports/runtime/eryx-python-proof-spike-1-report.md +0 -96
  617. package/evals/reports/runtime/eryx-python-proof-spike-2-report.md +0 -151
  618. package/evals/reports/runtime/eryx-python-script-derive-execution-1-report.md +0 -233
  619. package/evals/reports/runtime/jq-script-derive-execution-1-report.md +0 -215
  620. package/evals/reports/runtime/jq-wasm-proof-spike-1-report.md +0 -67
  621. package/evals/reports/runtime/output-router-benchmark-1-report.md +0 -81
  622. package/evals/reports/runtime/output-router-capture-derive-provider-eval-1-report.md +0 -35
  623. package/evals/reports/runtime/output-router-codex-large-index-sqlite-adhoc-report.md +0 -150
  624. package/evals/reports/runtime/output-router-codex-qa-benchmark-1-report.md +0 -46
  625. package/evals/reports/runtime/output-router-command-benchmark-1-report.md +0 -62
  626. package/evals/reports/runtime/output-router-index-benchmark-1-report.md +0 -64
  627. package/evals/reports/runtime/output-router-regression-1-report.md +0 -81
  628. package/evals/reports/runtime/output-router-transform-eval-1-report.md +0 -28
  629. package/evals/reports/runtime/pi-observed-routing-eval-1-report.md +0 -44
  630. package/evals/reports/runtime/quickjs-script-derive-execution-1-report.md +0 -159
  631. package/evals/reports/runtime/quickjs-wasi-proof-spike-1-report.md +0 -55
  632. package/evals/reports/runtime/render-format-row-protocol-1-report.md +0 -58
  633. package/evals/reports/runtime/script-sandbox-adapter-selection-review-1-report.md +0 -84
  634. package/evals/reports/runtime/script-sandbox-probe-resource-hardening-1-report.md +0 -51
  635. package/evals/reports/runtime/storage-policy-benchmark-1-report.md +0 -176
  636. package/evals/reports/runtime/vault-index-storage-spike-1-report.md +0 -39
  637. package/evals/reports/runtime/workflow-context-hook-1-report.md +0 -82
  638. package/evals/runbooks/adversarial-eval-runbook.md +0 -28
  639. package/evals/runbooks/fixture-eval-runbook.md +0 -137
  640. package/evals/runbooks/handoff-eval-runbook.md +0 -22
  641. package/evals/runbooks/verify-work-eval-runbook.md +0 -27
  642. package/evals/scripts/audit-command-surface.sh +0 -160
  643. package/evals/scripts/check-activation-contract.sh +0 -121
  644. package/evals/scripts/check-runtime-context-hook.sh +0 -183
  645. package/evals/scripts/grade-fixture-eval.sh +0 -446
  646. package/evals/scripts/run-claude-fixture-eval.sh +0 -26
  647. package/evals/scripts/run-codex-fixture-eval.sh +0 -26
  648. package/evals/scripts/run-eryx-python-proof-spike.js +0 -559
  649. package/evals/scripts/run-fixture-eval-by-id.sh +0 -23
  650. package/evals/scripts/run-fixture-eval.sh +0 -464
  651. package/evals/scripts/run-jq-wasm-proof-spike.js +0 -299
  652. package/evals/scripts/run-output-router-transform-eval.js +0 -115
  653. package/evals/scripts/run-pi-observed-routing-eval.js +0 -478
  654. package/evals/scripts/run-quickjs-wasi-proof-spike.js +0 -339
  655. package/evals/scripts/skill-evidence.sh +0 -316
  656. package/evals/scripts/validate-release-metadata.sh +0 -486
  657. package/evals/suites/v0.1-acceptance-suite.md +0 -131
  658. package/pi-extension/dist/host-producer-identification.js +0 -186
  659. package/pi-extension/dist/mcp-config.js +0 -30
  660. package/pi-extension/dist/native-safety-net.js +0 -209
  661. package/pi-extension/dist/observed-tool-routing.js +0 -114
  662. package/pi-extension/dist/renderers.js +0 -835
  663. package/pi-extension/dist/router-tools.js +0 -438
  664. package/pi-extension/dist/runtime-context.js +0 -288
  665. package/pi-extension/dist/schemas.js +0 -254
  666. package/pi-extension/dist/status.js +0 -449
  667. package/pi-extension/dist/utils.js +0 -467
  668. package/pi-extension/src/runtime-context.ts +0 -347
  669. package/pi-extension/src/status.ts +0 -505
  670. package/plugin-docs/README.md +0 -40
  671. package/plugin-docs/adr/0001-three-modes.md +0 -27
  672. package/plugin-docs/adr/0002-evals-before-hooks.md +0 -17
  673. package/plugin-docs/adr/0003-release-boundary.md +0 -21
  674. package/plugin-docs/adr/0004-discover-replaces-shallow-discovery-skills.md +0 -26
  675. package/plugin-docs/adr/README.md +0 -10
  676. package/plugin-docs/architecture.md +0 -89
  677. package/plugin-docs/output-router.md +0 -396
  678. package/plugin-docs/release-evidence.md +0 -77
  679. package/plugin-docs/skills.md +0 -44
  680. package/plugin-docs/workflow.md +0 -57
  681. package/skills/diagnose-failure/references/feedback-loop-catalog.md +0 -25
  682. package/skills/discover/ARTIFACT-DESTINATIONS.md +0 -58
  683. package/skills/discover/CHECKPOINTS.md +0 -74
  684. package/skills/discover/SKILL.md +0 -173
  685. package/skills/evaluate-skill/references/eval-patterns.md +0 -41
  686. package/skills/evaluate-skill/references/grading-priority.md +0 -33
  687. package/skills/execute-plan/SKILL.md +0 -133
  688. package/skills/execute-plan/references/execution-map.md +0 -122
  689. package/skills/interview-gate/SKILL.md +0 -90
  690. package/skills/setup-freeflow/references/output-router-setup.md +0 -102
  691. package/skills/workflow/references/workflow-map.md +0 -104
  692. /package/pi-extension/src/{mcp-config.ts → integrations/mcp-config.ts} +0 -0
@@ -1,57 +0,0 @@
1
- # Workflow
2
-
3
- Freeflow is a workflow layer, not a new agent. It helps the agent choose the right amount of process for the task.
4
-
5
- ## Modes
6
-
7
- - `conversation`: non-mutating discussion, read-only exploration, and planning in chat; edits or other state-changing work require switching to `workflow` or `strict-workflow` first.
8
- - `workflow`: default for consequential work; use the workflow spine and scale detail to risk.
9
- - `strict-workflow`: high-risk or hard-to-reverse work with stronger gates.
10
-
11
- Use strict-workflow for security, privacy, billing, public APIs, migrations, data loss, compatibility, permissions, deployment, or irreversible architecture.
12
-
13
- ## Map
14
-
15
- ```mermaid
16
- flowchart LR
17
- Request{request}
18
- Talk[conversation<br/>answer directly]
19
- Discover[discover<br/>checkpoint]
20
- Spec[spec]
21
- Plan[plan]
22
- Build[build<br/>execute / diagnose]
23
- Check[review + verify]
24
- Close[commit / handoff]
25
-
26
- Request -->|question| Talk
27
- Request -->|work| Discover
28
- Discover --> Spec --> Plan --> Build --> Check --> Close
29
- Check -. new evidence or failed check .-> Discover
30
- ```
31
-
32
- The map is orienting, not mandatory. Small reversible work can skip unnecessary artifacts and gates. Use `/discover` for the discovery loop.
33
-
34
- ## Backward Edge
35
-
36
- Loop back when new evidence changes the path:
37
-
38
- ```mermaid
39
- flowchart LR
40
- Trigger{new evidence<br/>conflict<br/>failed check<br/>owner decision}
41
- Decide{changes<br/>next action?}
42
- Continue[continue]
43
- Reenter[clarify / discover<br/>revise spec or plan<br/>diagnose / split / stop]
44
-
45
- Trigger --> Decide
46
- Decide -->|no| Continue
47
- Decide -->|yes| Reenter
48
- Reenter -. resume .-> Decide
49
- ```
50
-
51
- The agent should not silently choose the backward destination when the choice changes product behavior, scope, compatibility, public APIs, security, privacy, billing, data loss, permissions, or irreversible architecture.
52
-
53
- ## Bypass
54
-
55
- Bypass skips ceremony, not judgment.
56
-
57
- Use `bypass` only to skip an unnecessary gate. It does not skip user-owned decisions, source-truth conflicts, risky checks, or fresh verification before completion claims.
@@ -1,25 +0,0 @@
1
- # Feedback Loop Catalog
2
-
3
- Pick the smallest loop that proves the user's failure. Prefer loops the agent can run repeatedly.
4
-
5
- ## Common Loops
6
-
7
- - Failing test: best when an existing seam reaches the bug.
8
- - Repro command: CLI, script, package task, or one exact command with fixture input.
9
- - HTTP script: one request or short request sequence against a local service.
10
- - Browser script: UI flow with DOM, console, and network assertions.
11
- - Captured trace: HAR, log slice, event payload, seed, screenshot timestamp, core dump, or production sample replayed locally.
12
- - Throwaway harness: minimal script around the real module when no test framework exists.
13
- - Differential loop: same input through old versus new code, config, dataset, or dependency version.
14
- - Instrumented run: targeted logs or counters at boundaries that distinguish hypotheses.
15
-
16
- ## Loop Quality
17
-
18
- Improve the loop before fixing:
19
-
20
- - Faster: remove unrelated startup and setup.
21
- - Sharper: assert the exact symptom, not generic success.
22
- - More deterministic: pin time, seed randomness, isolate network and filesystem.
23
- - More representative: use the reported input shape and environment boundary.
24
-
25
- If none is possible, stop and ask for the smallest missing artifact: command, logs, trace, failing input, screenshot with timestamp, environment access, or permission for temporary instrumentation.
@@ -1,58 +0,0 @@
1
- # Artifact Destinations
2
-
3
- Use this when a discovery checkpoint or decision must survive beyond chat.
4
-
5
- ## Destination Test
6
-
7
- Before writing, answer:
8
-
9
- - What needs to survive?
10
- - Who will treat the destination as authority?
11
- - Which existing artifact already owns this kind of truth?
12
- - Would writing here create a new convention?
13
- - Would a future agent know to look here?
14
-
15
- If the owner is unclear, ask before writing.
16
-
17
- ## Common Destinations
18
-
19
- | Discovery output | Prefer | Avoid |
20
- | --- | --- | --- |
21
- | Domain term or meaning | Existing `CONTEXT.md`, glossary, or domain doc | README fallback; creating domain memory without asking |
22
- | Feature behavior, scope, acceptance | Existing or requested spec | ADR unless the ADR test passes |
23
- | Future implementation path | Plan | Hidden plan inside a discovery note |
24
- | Evidence summary before requirements settle | Existing discovery, research, or design notes; otherwise ask | Creating `docs/research/` by default |
25
- | Immediate continuation state | Handoff | Durable decision memory |
26
- | Hard-to-reverse surprising tradeoff from alternatives | ADR | Product note that hides rationale |
27
- | Rejected approach | ADR if tradeoff-heavy; otherwise spec/decision note | Changelog or README by default |
28
- | No durable value | Chat checkpoint | New file for ceremony |
29
-
30
- ## ADR Test
31
-
32
- Use an ADR only when all are true:
33
-
34
- - Hard to reverse.
35
- - Surprising without context.
36
- - Chosen from real alternatives.
37
-
38
- Otherwise use the owning spec, decision note, domain doc, handoff, or chat.
39
-
40
- ## Stop Rules
41
-
42
- Do not create a new `CONTEXT.md`, glossary, `docs/decisions/`, `docs/specs/`, `docs/research/`, or ADR folder just because the checkpoint needs memory.
43
-
44
- A missing destination convention is a decision point.
45
-
46
- Do not put volatile file lists, command output, temporary TODOs, or implementation inventory in durable decision memory. Put continuation state in a handoff or plan when needed.
47
-
48
- ## Question Shape
49
-
50
- When destination is ambiguous, ask one direct question:
51
-
52
- ```text
53
- I have a checkpoint worth preserving: [what].
54
- The narrowest destination is [recommended artifact] because [why].
55
- Should I save it there?
56
- ```
57
-
58
- Do not write the artifact before the user chooses the destination.
@@ -1,74 +0,0 @@
1
- # Discovery Checkpoints
2
-
3
- A checkpoint is the output of discovery. It preserves the current understanding and names the next route.
4
-
5
- Discovery does not end by silently writing a spec, plan, or implementation.
6
-
7
- ## Chat Or Artifact
8
-
9
- Keep the checkpoint in chat when the work is short-lived and the next turn can safely continue from conversation context.
10
-
11
- Save the checkpoint when it must guide later work, another agent, a future session, or a durable artifact.
12
-
13
- Use the narrowest owning artifact:
14
-
15
- - Spec: behavior, scope, requirements, acceptance, product decisions.
16
- - Plan: future implementation path and checks.
17
- - Handoff: immediate continuation state.
18
- - Decision note or ADR: durable tradeoff or rejected approach.
19
- - Domain memory: stable term or domain meaning.
20
- - Discovery note: evidence summary that must be reused before requirements are settled.
21
-
22
- If no clear destination exists, ask before writing.
23
-
24
- ## Shape
25
-
26
- Use this shape. Omit sections that do not apply.
27
-
28
- ```text
29
- Question / goal: ...
30
- Evidence: ...
31
- Current understanding: ...
32
- Settled: ...
33
- Tentative: ...
34
- Open: ...
35
- Recommendation: ...
36
- Next: Forward | Backward | Branch | Stop — ...
37
- ```
38
-
39
- ## Good Checkpoints
40
-
41
- A good checkpoint:
42
-
43
- - Links or names live evidence instead of copying volatile inventory.
44
- - Separates settled facts from tentative assumptions.
45
- - Names open decisions that change the next route.
46
- - Recommends the next route without treating it as permission.
47
- - Avoids hiding user-owned decisions in polished prose.
48
-
49
- ## Bad Checkpoints
50
-
51
- Bad:
52
-
53
- ```text
54
- We discussed a lot. I'll implement the recommended path.
55
- ```
56
-
57
- because it skips settled/tentative/open state and jumps to action.
58
-
59
- Bad:
60
-
61
- ```text
62
- Decision: create docs/decisions/ and record everything.
63
- ```
64
-
65
- when the repo has no decision-note convention and the user has not chosen that destination.
66
-
67
- Good:
68
-
69
- ```text
70
- Settled: RiskReview is manual today; $500+ invoices enter review.
71
- Tentative: retry scheduling should assist support, not replace review.
72
- Open: should this be support-assistive or fully automated?
73
- Next: Backward — answer the automation-scope decision before spec.
74
- ```
@@ -1,173 +0,0 @@
1
- ---
2
- name: discover
3
- description: Use before the first answer or repo/code exploration when the user is shaping an idea, feature, tool/API surface, runtime behavior, architecture direction, vague task, bug-like unknown, or consequential hypothesis/question before spec, plan, build, or durable memory. Trigger especially on "should we", "why not", "what do you think", or design questions.
4
- ---
5
-
6
- # Discover
7
-
8
- Discover is the context-building loop before action: evidence-guided discussion that finds the next high-leverage decision and ends in a checkpoint.
9
-
10
- If this skill matches, load it before the first repo search, code read, web lookup, command, design answer, or artifact. Direct questions still use discovery-light: inspect the smallest relevant evidence, then answer directly.
11
-
12
- Priority: Interview Gate stops silent decisions and user-owned decisions first. Discover handles context-building after no immediate stop condition remains. Output Router only chooses evidence transport after the workflow/discovery route is clear.
13
-
14
- Do not turn discovery into a questionnaire, a report by default, or a decision-capture script. The hard part is shared understanding, not filling an artifact.
15
-
16
- For short-lived work, keep the checkpoint in chat. When it must guide later work, save it in the narrowest owning artifact: spec, plan, handoff, decision note, ADR, domain memory, or discovery note.
17
-
18
- Goal: enough evidence and shared understanding to route to spec, plan, more discovery, durable memory, build, or stop without inventing product, domain, scope, public API, security, privacy, billing, data-loss, compatibility, or architecture decisions.
19
-
20
- Hard stop: do not satisfy requests for “every question,” “grill me,” “25-50 questions,” or recommended answers for each question. That creates an approval script. Inspect evidence, name the decision tree, and ask the next one or two path-changing questions instead.
21
-
22
- ## Evidence Before Questions
23
-
24
- Inspect first when the answer is discoverable:
25
-
26
- - code, tests, docs, ADRs, specs, policies, issues, logs, traces, or current repo state;
27
- - provided files, URLs, screenshots, transcripts, or user context;
28
- - current external docs or web sources when framework/library facts, ecosystem behavior, or recent changes matter.
29
-
30
- Ask before inspecting only when multiple targets would change the discovery path or when access/scope is user-owned.
31
-
32
- For a direct question, keep the loop light:
33
-
34
- ```text
35
- question -> inspect smallest relevant evidence -> answer directly
36
- ```
37
-
38
- No artifact, plan, or checkpoint is required unless the answer changes future consequential work.
39
-
40
- ## Design Pressure
41
-
42
- Use `../design-for-depth/SKILL.md` when discovery shapes modules, interfaces, seams, adapters, architecture direction, growing scope, codebase exploration, or repeated review/execution churn.
43
-
44
- Do not decide architecture up front just because the lens is active. Use it to notice whether complexity is being hidden behind useful interfaces or spread across callers, tests, docs, artifacts, and future agents.
45
-
46
- Use the deletion test on questions and decisions: if skipping it makes complexity spread downstream, it matters. If skipping it makes complexity vanish, it was ceremony.
47
-
48
- ## Scope Framing
49
-
50
- Preserve the user's intended scope. Do not silently turn a large idea into `v1`/`v2`, MVP, release, or roadmap framing.
51
-
52
- Use implementation slices, implementation phases, checklists, or multiple artifacts to make large work understandable and executable without changing what the user asked for.
53
-
54
- Versioned shipping is a product/scope decision. Use it only when the user asks for release planning, the source context already uses that framing, or you have named why release boundaries matter and the user agrees.
55
-
56
- If the full scope is too large for one safe pass, say that directly. Recommend implementation slices or artifact splits first; recommend versioned releases only when that is the user's actual problem.
57
-
58
- ## Loop
59
-
60
- 1. Identify the target: repo area, artifact, URL, product, model, package, paper, issue, symptom, or idea. If multiple plausible targets would change discovery, ask for the source.
61
- 2. Inspect the smallest evidence that can change the current route.
62
- 3. Explain the current understanding in plain language.
63
- 4. Brainstorm or compare paths when the direction is still forming; do not force a decision too early.
64
- 5. Ask one path-changing question when the answer would change the next artifact or action.
65
- 6. Let the user challenge, counter-question, or add constraints. Treat that as discovery input, not interruption.
66
- 7. Refresh evidence when new claims, conflicts, or branches appear.
67
- 8. Track decisions as settled, tentative, or open.
68
- 9. Stop when remaining ambiguity would not change the next route.
69
-
70
- Discovery may move through evidence, brainstorming, targeted questioning, design, and checkpointing in any order. Do not force a fixed sequence when the evidence or conversation needs a loop.
71
-
72
- ## Final Question Pass
73
-
74
- Before routing from discovery to `write-spec`, run a final ambiguity sweep.
75
-
76
- Trigger it when:
77
-
78
- - the user asks to write the spec;
79
- - the user says brainstorming, discovery, or discussion feels complete;
80
- - the user asks to move to the next phase;
81
- - you are about to close discovery with `Next: Forward — write-spec`.
82
-
83
- Ask the smallest set of remaining path-changing questions whose answers would affect spec scope, product behavior, acceptance, public API, architecture, security, privacy, billing, data-loss, compatibility, or artifact destination.
84
-
85
- If meaningful ambiguity remains, route with `Next: Forward — final question pass before write-spec` and ask those questions. If no meaningful questions remain, say so and route to `write-spec`.
86
-
87
- Do not use this while brainstorming, research, or comparison is still active. Do not turn it into a long questionnaire or recommendation approval script.
88
-
89
- ## Question Discipline
90
-
91
- Prefer deeper questions over more questions. Ask as many as the situation needs, but each question should earn its place.
92
-
93
- Before asking, run the deletion test:
94
-
95
- ```text
96
- If I do not ask this, does downstream complexity spread?
97
- ```
98
-
99
- If no, skip it.
100
-
101
- Bad question:
102
-
103
- ```text
104
- Should the button be blue or green?
105
- ```
106
-
107
- when the feature purpose is still unclear.
108
-
109
- Good question:
110
-
111
- ```text
112
- Is this for one workflow or many workflows?
113
- ```
114
-
115
- because it changes the module interface, spec shape, and test surface.
116
-
117
- Long question chains are a failure mode, not diligence. Never number out a long list just because the user asked for one.
118
-
119
- Do not turn recommendations into an approval script. Recommendations should clarify tradeoffs, not make it easy to rubber-stamp decisions. Give the recommendation for the current path-changing question, then wait when the choice is user-owned.
120
-
121
- After a few path-changing questions, or whenever the user introduces a new claim, inspect evidence, summarize, or checkpoint before asking more.
122
-
123
- ## Decision Discipline
124
-
125
- Read `ARTIFACT-DESTINATIONS.md` when a checkpoint or decision must survive beyond chat and the owning destination is not obvious.
126
-
127
- Not every answer is durable. Keep a lightweight ledger:
128
-
129
- - **Settled** — explicit user decision or evidence-backed fact.
130
- - **Tentative** — useful working assumption, not authority.
131
- - **Open** — unresolved question that can change the next route.
132
-
133
- Persist a decision only when absence would make future work rediscover or contradict it.
134
-
135
- Put durable decisions where they belong:
136
-
137
- - Domain term or meaning -> existing glossary, `CONTEXT.md`, or domain doc.
138
- - Feature behavior, scope, or acceptance -> spec.
139
- - Future implementation path -> plan.
140
- - Immediate continuation state -> handoff.
141
- - Hard-to-reverse, surprising tradeoff chosen from alternatives -> ADR.
142
- - No durable value -> leave it in chat.
143
-
144
- If the destination is missing or ambiguous, ask before writing.
145
-
146
- ## Stop Conditions
147
-
148
- Stop and ask when discovery reveals a user-owned decision, source-of-truth conflict, path conflict, material method substitution, or ambiguous artifact destination.
149
-
150
- Do not write implementation files from discovery.
151
-
152
- Write specs, plans, ADRs, handoffs, or decision notes only when the user asks or the checkpoint must guide later work.
153
-
154
- Do not leave discovery by silently jumping to spec, plan, or build. Before `write-spec`, run the final question pass. Close with a checkpoint when consequential direction changed.
155
-
156
- ## Checkpoint Shape
157
-
158
- Read `CHECKPOINTS.md` when saving a checkpoint or choosing chat versus artifact.
159
-
160
- Use this shape when ending a discovery pass or routing forward. Save it in the owning artifact when it must survive beyond chat. Omit sections that do not apply.
161
-
162
- ```text
163
- Question / goal: ...
164
- Evidence: ...
165
- Current understanding: ...
166
- Settled: ...
167
- Tentative: ...
168
- Open: ...
169
- Recommendation: ...
170
- Next: Forward | Backward | Branch | Stop — ...
171
- ```
172
-
173
- `Next:` is routing, not permission to create the next artifact.
@@ -1,41 +0,0 @@
1
- # Eval Patterns
2
-
3
- Choose the smallest repeatable artifact that preserves the failure and can be graded later.
4
-
5
- ## Common Shapes
6
-
7
- Use a prompt/pass-criteria update when the failure is already captured and the missing piece is expected behavior.
8
-
9
- Use a transcript note when the failure is conversational: answering vs artifact creation, clarification, refusal, routing, or command interpretation.
10
-
11
- Use a fixture eval when files, repo evidence, commands, state files, installed memory, or generated artifacts affect the outcome.
12
-
13
- Use saved-run grading when the job is to judge existing outputs, diffs, logs, or transcripts. Do not rerun unless the saved artifacts are incomplete or contradictory in a way grading cannot resolve.
14
-
15
- Use separate baseline and with-skill fixtures when the tested behavior depends on installed memory, setup output, host config, or pre-existing state.
16
-
17
- ## Comparison Choices
18
-
19
- Use no-skill baseline versus with-skill when proving the skill adds value over the ordinary agent or installed memory-free behavior.
20
-
21
- Use previous skill version versus updated skill when testing a wording change, runtime context change, or regression fix for a skill that already exists.
22
-
23
- For previous-version comparisons, save the exact old skill source, commit hash, plugin version, or temp copy path used as the control. Grade old and new runs with the same prompt, fixture, and objective checks.
24
-
25
- ## Harness Discipline
26
-
27
- Use the repo's runner when one exists. Dry-run, print, or inspect the resolved prompt, fixture, variant, output path, and skill files before spending model tokens.
28
-
29
- Do not build a harness because the user says "evaluate." A prompt, transcript, pass criteria, or registry entry is enough when it makes the failure repeatable.
30
-
31
- "Do not add a harness" means avoid machinery. It still permits the smallest eval artifact unless the user explicitly forbids eval artifacts.
32
-
33
- Save the final response and diff for fixture runs. Save command output or git status when those are the grading surface.
34
-
35
- ## Weak Eval Signs
36
-
37
- - Baseline and with-skill both pass without meaningful behavioral difference.
38
- - The prompt tells the agent the intended answer instead of creating pressure.
39
- - The only grade is the agent's self-assessment.
40
- - The eval cannot show whether files changed.
41
- - The fixture forbids behavior the repo actually allows.
@@ -1,33 +0,0 @@
1
- # Grading Priority
2
-
3
- Grade artifacts before explanations.
4
-
5
- Evidence order:
6
-
7
- 1. Filesystem diff, git diff, created/deleted files, and git state.
8
- 2. Command output, exit codes, logs, traces, screenshots, or saved reports.
9
- 3. Final response.
10
- 4. Full transcript.
11
- 5. Agent self-assessment.
12
-
13
- If lower-priority evidence conflicts with higher-priority evidence, the higher-priority evidence wins.
14
-
15
- ## Pass And Fail Rules
16
-
17
- A run passes only when the artifacts satisfy the eval assertions. A polished final response cannot repair a bad diff.
18
-
19
- If a final response claims an eval artifact was created but the diff shows only a skill edit, grade it as missing eval-first evidence.
20
-
21
- If the diff shows a prohibited file edit, grade the run by the diff even when the final response says no files changed.
22
-
23
- If the outcome can be checked by line count, file existence, JSON parsing, git status, or diff content, check it mechanically.
24
-
25
- Use transcript review only when the behavior cannot be judged from saved files, command output, or final response.
26
-
27
- ## Rerun Rules
28
-
29
- Rerun the failed side first after changing skill wording.
30
-
31
- Rerun both sides only when the prompt, fixture, harness, or grading criteria changed enough to invalidate the comparison.
32
-
33
- Do not rerun to make a failing saved artifact look better. Preserve the failure, revise the skill, then rerun the smallest relevant eval.
@@ -1,133 +0,0 @@
1
- ---
2
- name: execute-plan
3
- description: Use when implementing an approved plan, executing planned slices, resuming planned work, handling planned verification failures, adjudicating review findings during execution, or encountering scope/source conflicts while carrying out a plan.
4
- ---
5
-
6
- # Execute Plan
7
-
8
- Execute the plan. Do not improve it silently.
9
-
10
- The plan is instructions, not authority. Live repo evidence and source truth win.
11
-
12
- Move one verified slice at a time. Preserve rollback, reviewability, and user control.
13
-
14
- Read `references/execution-map.md` when the plan has multiple slices, TDD/benchmark work, review checkpoints, per-slice commits, context-window pressure, or any failed check/review/source-scope conflict.
15
-
16
- Use `../design-for-depth/SKILL.md` when a slice changes modules, interfaces, seams, adapters, architecture, or when review findings become a stream of edge-case patches. Ask whether the slice is deepening the module or spreading complexity across callers, tests, docs, or review comments.
17
-
18
- ## Classify First
19
-
20
- - Valid plan: inspect source context, execute the next slice, verify it.
21
- - Plan/source conflict: stop and ask which source should change.
22
- - Hidden owner decision: stop and ask.
23
- - Missing verification: stop before consequential edits and ask to revise the plan or approve a diagnostic path.
24
- - Missing plan: ask for a plan or route to `write-plan`.
25
- - Scope expansion: stop before absorbing it; route back to discover, spec, or plan.
26
- - Review failure during execution: classify findings and report the route before editing from them.
27
-
28
- ## Before Editing
29
-
30
- Read:
31
-
32
- - The plan.
33
- - The source spec or requirements the plan cites.
34
- - Relevant docs, tests, policies, ADRs, and code.
35
- - Handoffs only as memory.
36
-
37
- Live repo evidence overrides stale plans and handoffs.
38
-
39
- Before each non-trivial slice, name the slice contract:
40
-
41
- ```text
42
- Slice:
43
- Source truth:
44
- Module/interface changed:
45
- Behavior/test/benchmark:
46
- Verification:
47
- Review checkpoint:
48
- Commit or handoff checkpoint:
49
- Stop conditions:
50
- ```
51
-
52
- Do not start a slice if the remaining context is not enough to orient, edit, verify, and checkpoint it. Stop with a handoff route instead.
53
-
54
- Do not execute a plan that would:
55
-
56
- - Invent or change product behavior, scope, domain meaning, compatibility, public API behavior, security, privacy, billing, data-loss, or architecture.
57
- - Override docs, tests, specs, policies, ADRs, or established behavior.
58
- - Treat "do not ask", "just execute", "latest context", or "handoff says" as conflict approval.
59
- - Skip verification for consequential behavior.
60
- - Rewrite a verification script, test, source-truth doc, policy, or spec to make the plan pass unless the plan explicitly authorized that artifact change and it matches source truth.
61
-
62
- Name the conflict or missing decision. Ask which path to follow. Recommend the path supported by evidence.
63
-
64
- For source-truth conflicts or missing verification, the final line must be a direct choice question.
65
-
66
- For missing verification, ask whether to revise the plan to add a check or approve a specific verification path.
67
-
68
- ## Slice Execution
69
-
70
- Work in vertical slices.
71
-
72
- For each slice:
73
-
74
- - Make only the edits needed for that slice.
75
- - Keep the slice boundary intact; do not blend future slices into the current one for convenience.
76
- - Run the planned check or the smallest equivalent check.
77
- - If the check fails, stop and report the evidence before changing direction.
78
- - If new evidence invalidates the plan, stop before patching forward.
79
- - If implementation spreads policy, edge cases, or verification across callers/tests/docs, treat it as design pressure before adding more patches.
80
-
81
- When TDD is requested by the user, plan, or repo practice, use TDD inside the slice:
82
-
83
- ```text
84
- one behavior test or benchmark
85
- -> minimal implementation
86
- -> refactor while green
87
- -> verify the slice
88
- ```
89
-
90
- Do not write all tests first and all implementation later. Do not anticipate future slices.
91
-
92
- After a planned verification command fails, do not edit the verifier, tests, docs, policy, source-truth files, or unrelated code to make it pass. This is true even when the verifier appears wrong and even when the user said to "fix whatever is needed". A bad verifier is a plan defect; stop, report the failing command and conflicting evidence, and ask whether to revise the plan or change the source truth.
93
-
94
- Local reversible implementation details can be chosen from repo conventions.
95
-
96
- If per-slice commits are requested, do not continue to the next slice with a verified slice still uncommitted. Route to `commit-work` after review/verification evidence exists.
97
-
98
- ## Review Checkpoints
99
-
100
- Use review where the plan, risk, or slice boundary calls for it. Do not review every slice by habit.
101
-
102
- A non-passing review is a phase exit, not an autonomous patch loop.
103
-
104
- When a review returns findings during execution:
105
-
106
- 1. Inspect the relevant code, tests, docs, plan, and source truth.
107
- 2. Classify each material finding: accepted, rejected, question, or needs evidence.
108
- 3. Report the route before editing from that review batch.
109
-
110
- The turn that receives a non-passing review ends with adjudication and route only. Do not edit from that review batch in the same turn, even when the user or reviewer says to apply all findings and continue reviewing.
111
-
112
- Do not immediately apply findings and request another review in the same loop. Do not treat non-blocking findings or reviewer questions as automatic failure.
113
-
114
- If all accepted findings are small, in-scope, and supported by source truth, recommend a bounded fix pass as the next route. Do not perform that fix pass until the user or parent explicitly chooses it after seeing the adjudication. If findings change scope, behavior, source truth, public API, security, privacy, billing, data loss, compatibility, or architecture, route backward before editing.
115
-
116
- Three review passes is the hard cap for the same slice/work scope. At pass 3, stop, classify, and diagnose. Do not request a fourth broad review. Use a `diagnose-failure`-style loop to decide whether the issue is shallow discovery, weak spec, wrong plan slice, ambiguous policy, source-truth conflict, shallow module/interface, implementation bug, or stale reviewer context.
117
-
118
- ## Backward Edge
119
-
120
- If implementation reveals that the plan is wrong, incomplete, or too narrow, re-enter discover/spec/plan before continuing.
121
-
122
- Do not absorb material scope expansion into execution. New behavior, API shape, security/privacy/billing/data-loss decisions, compatibility changes, or irreversible architecture usually need a revised spec or plan.
123
-
124
- ## Completion
125
-
126
- Report:
127
-
128
- - What changed.
129
- - What was verified.
130
- - Review status and any adjudicated findings.
131
- - Commit or handoff status when relevant.
132
- - What remains unverified.
133
- - Any blocked decisions or plan changes needed.