@hassangameryt/freeflow 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (639) hide show
  1. package/.agents/plugins/marketplace.json +1 -1
  2. package/.claude-plugin/marketplace.json +2 -2
  3. package/{plugins/freeflow/.claude-plugin → .claude-plugin}/plugin.json +1 -1
  4. package/{plugins/freeflow/.codex-plugin → .codex-plugin}/plugin.json +1 -1
  5. package/README.md +19 -14
  6. package/assets/freeflow-plugin-icon.png +0 -0
  7. package/{plugins/freeflow/command-surface.json → command-surface.json} +10 -2
  8. package/evals/README.md +32 -0
  9. package/evals/fixtures/copy-conventions.md +31 -0
  10. package/evals/fixtures/output-router/README.md +12 -0
  11. package/evals/fixtures/output-router/failed-command-output.txt +9 -0
  12. package/evals/fixtures/output-router/large-router-manual.md +63 -0
  13. package/evals/fixtures/output-router/native-large-output.txt +40 -0
  14. package/evals/fixtures/output-router/noisy-test-output.txt +30 -0
  15. package/evals/fixtures/output-router/verification-output.txt +8 -0
  16. package/evals/fixtures/tiny-artifact-review-app/README.md +5 -0
  17. package/evals/fixtures/tiny-artifact-review-app/docs/billing-policy.md +10 -0
  18. package/evals/fixtures/tiny-artifact-review-app/docs/specs/billing-webhook-api-draft.md +21 -0
  19. package/evals/fixtures/tiny-artifact-review-app/docs/specs/failed-payment-grace-period.md +24 -0
  20. package/evals/fixtures/tiny-artifact-review-app/docs/specs/immediate-downgrade.md +17 -0
  21. package/evals/fixtures/tiny-artifact-review-app/tests/billing.test.md +8 -0
  22. package/evals/fixtures/tiny-commit-ambiguous-app/.freeflow-eval-setup.sh +15 -0
  23. package/evals/fixtures/tiny-commit-ambiguous-app/README.md +3 -0
  24. package/evals/fixtures/tiny-commit-ambiguous-app/src/billing.ts +5 -0
  25. package/evals/fixtures/tiny-commit-ambiguous-app/src/settings-page.ts +1 -0
  26. package/evals/fixtures/tiny-commit-ambiguous-app/tests/settings.test.md +5 -0
  27. package/evals/fixtures/tiny-commit-app/.freeflow-eval-setup.sh +14 -0
  28. package/evals/fixtures/tiny-commit-app/README.md +3 -0
  29. package/evals/fixtures/tiny-commit-app/docs/specs/settings-title.md +5 -0
  30. package/evals/fixtures/tiny-commit-app/src/billing.ts +5 -0
  31. package/evals/fixtures/tiny-commit-app/src/settings-page.ts +1 -0
  32. package/evals/fixtures/tiny-commit-app/tests/settings.test.md +5 -0
  33. package/evals/fixtures/tiny-commit-everything-app/.freeflow-eval-setup.sh +15 -0
  34. package/evals/fixtures/tiny-commit-everything-app/README.md +3 -0
  35. package/evals/fixtures/tiny-commit-everything-app/docs/specs/settings-title.md +5 -0
  36. package/evals/fixtures/tiny-commit-everything-app/src/billing.ts +5 -0
  37. package/evals/fixtures/tiny-commit-everything-app/src/settings-page.ts +1 -0
  38. package/evals/fixtures/tiny-commit-everything-app/tests/settings.test.md +5 -0
  39. package/evals/fixtures/tiny-commit-staged-app/.freeflow-eval-setup.sh +15 -0
  40. package/evals/fixtures/tiny-commit-staged-app/README.md +3 -0
  41. package/evals/fixtures/tiny-commit-staged-app/docs/specs/settings-title.md +5 -0
  42. package/evals/fixtures/tiny-commit-staged-app/src/billing.ts +5 -0
  43. package/evals/fixtures/tiny-commit-staged-app/src/settings-page.ts +1 -0
  44. package/evals/fixtures/tiny-commit-staged-app/tests/settings.test.md +5 -0
  45. package/evals/fixtures/tiny-commit-staged-mixed-app/.freeflow-eval-setup.sh +15 -0
  46. package/evals/fixtures/tiny-commit-staged-mixed-app/README.md +3 -0
  47. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/billing-policy.md +3 -0
  48. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/release-notes.md +3 -0
  49. package/evals/fixtures/tiny-commit-staged-mixed-app/src/billing.ts +5 -0
  50. package/evals/fixtures/tiny-dashboard-stale-app/README.md +11 -0
  51. package/evals/fixtures/tiny-dashboard-stale-app/src/api-client.ts +13 -0
  52. package/evals/fixtures/tiny-dashboard-stale-app/src/dashboard-data.ts +19 -0
  53. package/evals/fixtures/tiny-dashboard-stale-app/tests/dashboard-data.test.md +9 -0
  54. package/evals/fixtures/tiny-decision-destination-app/README.md +6 -0
  55. package/evals/fixtures/tiny-decision-destination-app/src/risk.ts +8 -0
  56. package/evals/fixtures/tiny-decision-destination-app/tests/risk.test.md +4 -0
  57. package/evals/fixtures/tiny-design-depth-app/README.md +15 -0
  58. package/evals/fixtures/tiny-design-depth-app/src/routes.ts +25 -0
  59. package/evals/fixtures/tiny-design-depth-app/src/sendEmail.ts +3 -0
  60. package/evals/fixtures/tiny-design-depth-app/src/sendSms.ts +3 -0
  61. package/evals/fixtures/tiny-design-depth-app/src/telemetry.ts +3 -0
  62. package/evals/fixtures/tiny-design-depth-app/tests/routes.test.md +12 -0
  63. package/evals/fixtures/tiny-eval-method-app/README.md +8 -0
  64. package/evals/fixtures/tiny-eval-method-app/evals/reports/review-pr-eval-result.md +5 -0
  65. package/evals/fixtures/tiny-eval-method-app/evals/reports/triage-question-failure.md +17 -0
  66. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.diff +11 -0
  67. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.md +3 -0
  68. package/evals/fixtures/tiny-eval-method-app/skills/review-pr/SKILL.md +10 -0
  69. package/evals/fixtures/tiny-eval-method-app/skills/triage-question/SKILL.md +10 -0
  70. package/evals/fixtures/tiny-execute-plan-app/README.md +5 -0
  71. package/evals/fixtures/tiny-execute-plan-app/checks/settings-and-billing-check.sh +5 -0
  72. package/evals/fixtures/tiny-execute-plan-app/docs/billing-policy.md +10 -0
  73. package/evals/fixtures/tiny-execute-plan-app/docs/plans/immediate-downgrade-plan.md +11 -0
  74. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-plan.md +12 -0
  75. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-with-bad-check.md +15 -0
  76. package/evals/fixtures/tiny-execute-plan-app/docs/plans/unverified-billing-plan.md +9 -0
  77. package/evals/fixtures/tiny-execute-plan-app/docs/specs/failed-payment-grace-period.md +9 -0
  78. package/evals/fixtures/tiny-execute-plan-app/docs/specs/settings-title.md +9 -0
  79. package/evals/fixtures/tiny-execute-plan-app/src/billing.ts +7 -0
  80. package/evals/fixtures/tiny-execute-plan-app/src/settings-page.ts +1 -0
  81. package/evals/fixtures/tiny-execute-plan-app/tests/billing.test.md +7 -0
  82. package/evals/fixtures/tiny-execute-plan-app/tests/settings.test.md +5 -0
  83. package/evals/fixtures/tiny-execute-review-loop-app/README.md +5 -0
  84. package/evals/fixtures/tiny-execute-review-loop-app/docs/billing-policy.md +9 -0
  85. package/evals/fixtures/tiny-execute-review-loop-app/docs/plans/settings-title-plan.md +12 -0
  86. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/previous-agent-summary.md +15 -0
  87. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/review-pass-1-non-pass.md +11 -0
  88. package/evals/fixtures/tiny-execute-review-loop-app/docs/specs/settings-title.md +9 -0
  89. package/evals/fixtures/tiny-execute-review-loop-app/src/billing.ts +7 -0
  90. package/evals/fixtures/tiny-execute-review-loop-app/src/settings-page.ts +1 -0
  91. package/evals/fixtures/tiny-execute-review-loop-app/tests/billing.test.md +7 -0
  92. package/evals/fixtures/tiny-execute-review-loop-app/tests/settings.test.md +5 -0
  93. package/evals/fixtures/tiny-flaky-session-app/README.md +10 -0
  94. package/evals/fixtures/tiny-flaky-session-app/src/session-timeout.ts +13 -0
  95. package/evals/fixtures/tiny-flaky-session-app/tests/session-timeout.test.md +8 -0
  96. package/evals/fixtures/tiny-handoff-app/README.md +5 -0
  97. package/evals/fixtures/tiny-handoff-app/docs/billing-policy.md +10 -0
  98. package/evals/fixtures/tiny-handoff-app/docs/handoffs/old-agent-handoff.md +8 -0
  99. package/evals/fixtures/tiny-handoff-app/docs/plans/billing-retry-plan.md +11 -0
  100. package/evals/fixtures/tiny-handoff-app/src/billing.ts +7 -0
  101. package/evals/fixtures/tiny-handoff-report-app/README.md +3 -0
  102. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.diff +1 -0
  103. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.md +1 -0
  104. package/evals/fixtures/tiny-handoff-report-app/evals/scores/hof-003-score.txt +4 -0
  105. package/evals/fixtures/tiny-plan-app/README.md +5 -0
  106. package/evals/fixtures/tiny-plan-app/docs/billing-policy.md +10 -0
  107. package/evals/fixtures/tiny-plan-app/docs/specs/failed-payment-grace-period.md +25 -0
  108. package/evals/fixtures/tiny-plan-app/src/billing.ts +17 -0
  109. package/evals/fixtures/tiny-plan-app/src/settings-page.ts +1 -0
  110. package/evals/fixtures/tiny-plan-app/tests/billing.test.md +8 -0
  111. package/evals/fixtures/tiny-plan-app/tests/settings.test.md +5 -0
  112. package/evals/fixtures/tiny-plan-strict-api-app/README.md +3 -0
  113. package/evals/fixtures/tiny-plan-strict-api-app/docs/billing-policy.md +5 -0
  114. package/evals/fixtures/tiny-plan-strict-api-app/docs/specs/billing-webhook-api.md +25 -0
  115. package/evals/fixtures/tiny-plan-strict-api-app/src/billing.ts +5 -0
  116. package/evals/fixtures/tiny-plan-strict-api-app/src/webhooks.ts +3 -0
  117. package/evals/fixtures/tiny-plan-strict-api-app/tests/billing-webhook.test.md +8 -0
  118. package/evals/fixtures/tiny-post-setup-invalid-config-app/.freeflow/config.json +3 -0
  119. package/evals/fixtures/tiny-post-setup-invalid-config-app/AGENTS.md +15 -0
  120. package/evals/fixtures/tiny-post-setup-invalid-config-app/README.md +16 -0
  121. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/auth-review.md +15 -0
  122. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/billing-policy.md +10 -0
  123. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/copy-conventions.md +15 -0
  124. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/auth-middleware.ts +12 -0
  125. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/billing.ts +20 -0
  126. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/login-copy.ts +1 -0
  127. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/settings-page.ts +1 -0
  128. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/auth-middleware.test.md +8 -0
  129. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/billing.test.md +8 -0
  130. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/copy.test.md +7 -0
  131. package/evals/fixtures/tiny-post-setup-missing-config-app/AGENTS.md +15 -0
  132. package/evals/fixtures/tiny-post-setup-missing-config-app/README.md +16 -0
  133. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/auth-review.md +15 -0
  134. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/billing-policy.md +10 -0
  135. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/copy-conventions.md +15 -0
  136. package/evals/fixtures/tiny-post-setup-missing-config-app/src/auth-middleware.ts +12 -0
  137. package/evals/fixtures/tiny-post-setup-missing-config-app/src/billing.ts +20 -0
  138. package/evals/fixtures/tiny-post-setup-missing-config-app/src/login-copy.ts +1 -0
  139. package/evals/fixtures/tiny-post-setup-missing-config-app/src/settings-page.ts +1 -0
  140. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/auth-middleware.test.md +8 -0
  141. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/billing.test.md +8 -0
  142. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/copy.test.md +7 -0
  143. package/evals/fixtures/tiny-post-setup-source-conflict-app/.freeflow/config.json +3 -0
  144. package/evals/fixtures/tiny-post-setup-source-conflict-app/AGENTS.md +15 -0
  145. package/evals/fixtures/tiny-post-setup-source-conflict-app/README.md +16 -0
  146. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/auth-review.md +15 -0
  147. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/billing-policy.md +10 -0
  148. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/copy-conventions.md +15 -0
  149. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/auth-middleware.ts +12 -0
  150. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/billing.ts +20 -0
  151. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/login-copy.ts +1 -0
  152. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/settings-page.ts +1 -0
  153. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/auth-middleware.test.md +8 -0
  154. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/billing.test.md +8 -0
  155. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/copy.test.md +7 -0
  156. package/evals/fixtures/tiny-report-performance-app/README.md +11 -0
  157. package/evals/fixtures/tiny-report-performance-app/docs/performance.md +10 -0
  158. package/evals/fixtures/tiny-report-performance-app/src/report-summary.ts +29 -0
  159. package/evals/fixtures/tiny-report-performance-app/tests/report-performance.test.md +8 -0
  160. package/evals/fixtures/tiny-review-request-app/README.md +3 -0
  161. package/evals/fixtures/tiny-review-request-app/docs/billing-policy.md +5 -0
  162. package/evals/fixtures/tiny-review-request-app/docs/plans/billing-webhook-api.md +19 -0
  163. package/evals/fixtures/tiny-review-request-app/docs/review/previous-agent-summary.md +20 -0
  164. package/evals/fixtures/tiny-review-request-app/docs/specs/billing-webhook-api.md +25 -0
  165. package/evals/fixtures/tiny-review-request-app/src/billing.ts +11 -0
  166. package/evals/fixtures/tiny-review-request-app/src/webhooks.ts +19 -0
  167. package/evals/fixtures/tiny-review-request-app/tests/billing-webhook.test.md +8 -0
  168. package/evals/fixtures/tiny-review-work-app/README.md +5 -0
  169. package/evals/fixtures/tiny-review-work-app/docs/billing-policy.md +9 -0
  170. package/evals/fixtures/tiny-review-work-app/docs/plans/settings-title-plan.md +12 -0
  171. package/evals/fixtures/tiny-review-work-app/docs/review/previous-agent-summary.md +15 -0
  172. package/evals/fixtures/tiny-review-work-app/docs/specs/settings-title.md +9 -0
  173. package/evals/fixtures/tiny-review-work-app/src/billing.ts +7 -0
  174. package/evals/fixtures/tiny-review-work-app/src/settings-page.ts +1 -0
  175. package/evals/fixtures/tiny-review-work-app/tests/billing.test.md +7 -0
  176. package/evals/fixtures/tiny-review-work-app/tests/settings.test.md +5 -0
  177. package/evals/fixtures/tiny-saas-app/README.md +16 -0
  178. package/evals/fixtures/tiny-saas-app/docs/auth-review.md +15 -0
  179. package/evals/fixtures/tiny-saas-app/docs/billing-policy.md +10 -0
  180. package/evals/fixtures/tiny-saas-app/docs/copy-conventions.md +15 -0
  181. package/evals/fixtures/tiny-saas-app/docs/specs/settings-title.md +13 -0
  182. package/evals/fixtures/tiny-saas-app/src/auth-middleware.ts +12 -0
  183. package/evals/fixtures/tiny-saas-app/src/billing.ts +20 -0
  184. package/evals/fixtures/tiny-saas-app/src/login-copy.ts +1 -0
  185. package/evals/fixtures/tiny-saas-app/src/settings-page.ts +1 -0
  186. package/evals/fixtures/tiny-saas-app/tests/auth-middleware.test.md +8 -0
  187. package/evals/fixtures/tiny-saas-app/tests/billing.test.md +8 -0
  188. package/evals/fixtures/tiny-saas-app/tests/copy.test.md +7 -0
  189. package/evals/fixtures/tiny-setup-agents-app/AGENTS.md +6 -0
  190. package/evals/fixtures/tiny-setup-both-app/AGENTS.md +5 -0
  191. package/evals/fixtures/tiny-setup-both-app/CLAUDE.md +5 -0
  192. package/evals/fixtures/tiny-setup-claude-app/CLAUDE.md +6 -0
  193. package/evals/fixtures/tiny-setup-codex-rules-app/.codex/rules/shell.rules +5 -0
  194. package/evals/fixtures/tiny-setup-codex-rules-app/AGENTS.md +6 -0
  195. package/evals/fixtures/tiny-setup-conflict-app/AGENTS.md +6 -0
  196. package/evals/fixtures/tiny-setup-existing-block-app/AGENTS.md +14 -0
  197. package/evals/fixtures/tiny-setup-four-rules-app/CLAUDE.md +6 -0
  198. package/evals/fixtures/tiny-skill-dev-app/README.md +10 -0
  199. package/evals/fixtures/tiny-skill-dev-app/evals/prompts/review-pr-001.txt +8 -0
  200. package/evals/fixtures/tiny-skill-dev-app/evals/reports/review-pr-failure.md +17 -0
  201. package/evals/fixtures/tiny-skill-dev-app/skills/review-pr/SKILL.md +13 -0
  202. package/evals/fixtures/tiny-team-app/README.md +5 -0
  203. package/evals/fixtures/tiny-team-app/docs/product.md +6 -0
  204. package/evals/fixtures/tiny-team-app/src/teams.ts +9 -0
  205. package/evals/fixtures/tiny-team-app/src/users.ts +5 -0
  206. package/evals/fixtures/tiny-team-app/tests/teams.test.md +7 -0
  207. package/evals/fixtures/tiny-verify-app/README.md +5 -0
  208. package/evals/fixtures/tiny-verify-app/src/settings-nav.ts +1 -0
  209. package/evals/fixtures/tiny-verify-app/src/settings-page.ts +1 -0
  210. package/evals/fixtures/tiny-verify-app/tests/settings.test.md +7 -0
  211. package/evals/prompts/aon-001.txt +2 -0
  212. package/evals/prompts/byp-001.txt +2 -0
  213. package/evals/prompts/byp-002.txt +2 -0
  214. package/evals/prompts/cap-001.txt +3 -0
  215. package/evals/prompts/cap-002.txt +3 -0
  216. package/evals/prompts/cmd-001.txt +3 -0
  217. package/evals/prompts/cmd-002.txt +3 -0
  218. package/evals/prompts/cmd-003.txt +6 -0
  219. package/evals/prompts/cmd-004.txt +5 -0
  220. package/evals/prompts/cmd-005.txt +5 -0
  221. package/evals/prompts/cmd-006.txt +5 -0
  222. package/evals/prompts/cmd-007.txt +5 -0
  223. package/evals/prompts/cmd-008.txt +5 -0
  224. package/evals/prompts/cmd-009.txt +8 -0
  225. package/evals/prompts/cmd-010.txt +5 -0
  226. package/evals/prompts/cmd-011.txt +5 -0
  227. package/evals/prompts/cmd-012.txt +5 -0
  228. package/evals/prompts/cmd-013.txt +5 -0
  229. package/evals/prompts/cmd-014.txt +7 -0
  230. package/evals/prompts/cmd-015.txt +7 -0
  231. package/evals/prompts/cmd-016.txt +5 -0
  232. package/evals/prompts/cmt-001.txt +1 -0
  233. package/evals/prompts/cmt-002.txt +1 -0
  234. package/evals/prompts/cmt-003.txt +1 -0
  235. package/evals/prompts/cmt-004.txt +1 -0
  236. package/evals/prompts/cmt-005.txt +1 -0
  237. package/evals/prompts/dfd-001.txt +3 -0
  238. package/evals/prompts/dia-001.txt +3 -0
  239. package/evals/prompts/dia-002.txt +3 -0
  240. package/evals/prompts/dia-003.txt +3 -0
  241. package/evals/prompts/dis-001.txt +5 -0
  242. package/evals/prompts/dis-002.txt +5 -0
  243. package/evals/prompts/esk-001.txt +5 -0
  244. package/evals/prompts/esk-002.txt +5 -0
  245. package/evals/prompts/esk-003.txt +5 -0
  246. package/evals/prompts/grc-001.txt +3 -0
  247. package/evals/prompts/hof-001.txt +3 -0
  248. package/evals/prompts/hof-002.txt +1 -0
  249. package/evals/prompts/hof-003.txt +1 -0
  250. package/evals/prompts/hof-004.txt +1 -0
  251. package/evals/prompts/hof-005.txt +1 -0
  252. package/evals/prompts/ivg-001.txt +1 -0
  253. package/evals/prompts/ivg-002.txt +11 -0
  254. package/evals/prompts/ivg-003.txt +1 -0
  255. package/evals/prompts/ivg-004.txt +1 -0
  256. package/evals/prompts/mode-001.txt +3 -0
  257. package/evals/prompts/mode-002.txt +1 -0
  258. package/evals/prompts/mode-003.txt +3 -0
  259. package/evals/prompts/mode-004.txt +3 -0
  260. package/evals/prompts/mode-005.txt +3 -0
  261. package/evals/prompts/mode-006.txt +3 -0
  262. package/evals/prompts/mode-007.txt +3 -0
  263. package/evals/prompts/otr-001.txt +3 -0
  264. package/evals/prompts/otr-002.txt +3 -0
  265. package/evals/prompts/rar-001.txt +5 -0
  266. package/evals/prompts/rar-002.txt +3 -0
  267. package/evals/prompts/rar-003.txt +6 -0
  268. package/evals/prompts/rar-004.txt +15 -0
  269. package/evals/prompts/rar-005.txt +10 -0
  270. package/evals/prompts/rbr-001.txt +3 -0
  271. package/evals/prompts/rbr-002.txt +3 -0
  272. package/evals/prompts/rbr-003.txt +7 -0
  273. package/evals/prompts/rev-001.txt +3 -0
  274. package/evals/prompts/rev-002.txt +5 -0
  275. package/evals/prompts/rev-003.txt +6 -0
  276. package/evals/prompts/rev-004.txt +1 -0
  277. package/evals/prompts/rev-005.txt +15 -0
  278. package/evals/prompts/rev-006.txt +5 -0
  279. package/evals/prompts/stp-001.txt +1 -0
  280. package/evals/prompts/stp-002.txt +1 -0
  281. package/evals/prompts/stp-003.txt +1 -0
  282. package/evals/prompts/stp-004.txt +1 -0
  283. package/evals/prompts/stp-005.txt +1 -0
  284. package/evals/prompts/stp-006.txt +1 -0
  285. package/evals/prompts/stp-007.txt +1 -0
  286. package/evals/prompts/stp-008.txt +1 -0
  287. package/evals/prompts/stp-009.txt +1 -0
  288. package/evals/prompts/stp-010.txt +1 -0
  289. package/evals/prompts/stp-011.txt +1 -0
  290. package/evals/prompts/vfy-003.txt +1 -0
  291. package/evals/prompts/wpl-001.txt +3 -0
  292. package/evals/prompts/wpl-002.txt +5 -0
  293. package/evals/prompts/wpl-003.txt +3 -0
  294. package/evals/prompts/wpl-004.txt +5 -0
  295. package/evals/prompts/wpl-005.txt +1 -0
  296. package/evals/prompts/wrc-001.txt +10 -0
  297. package/evals/prompts/wrc-002.txt +3 -0
  298. package/evals/prompts/wsk-001.txt +6 -0
  299. package/evals/prompts/wsp-001.txt +5 -0
  300. package/evals/prompts/wsp-002.txt +15 -0
  301. package/evals/prompts/wsp-003.txt +3 -0
  302. package/evals/prompts/wsp-004.txt +15 -0
  303. package/evals/prompts/wsp-005.txt +5 -0
  304. package/evals/prompts/wsp-006.txt +12 -0
  305. package/evals/prompts/wsp-007.txt +9 -0
  306. package/evals/prompts/wsp-008.txt +13 -0
  307. package/evals/prompts/xpl-001.txt +1 -0
  308. package/evals/prompts/xpl-002.txt +3 -0
  309. package/evals/prompts/xpl-003.txt +3 -0
  310. package/evals/prompts/xpl-004.txt +3 -0
  311. package/evals/prompts/xpl-005.txt +5 -0
  312. package/evals/prompts/xpl-006.txt +3 -0
  313. package/evals/registries/adversarial-evals.json +39 -0
  314. package/evals/registries/fixture-evals.json +2513 -0
  315. package/evals/registries/handoff-evals.json +74 -0
  316. package/evals/registries/legacy-evals.json +147 -0
  317. package/evals/registries/skill-evidence.json +193 -0
  318. package/evals/registries/verify-work-evals.json +46 -0
  319. package/evals/reports/acceptance/v0.1-acceptance-report.md +130 -0
  320. package/evals/reports/by-command-surface/command-surface-1-report.md +45 -0
  321. package/evals/reports/by-command-surface/command-surface-10-report.md +76 -0
  322. package/evals/reports/by-command-surface/command-surface-11-report.md +77 -0
  323. package/evals/reports/by-command-surface/command-surface-12-report.md +91 -0
  324. package/evals/reports/by-command-surface/command-surface-13-report.md +64 -0
  325. package/evals/reports/by-command-surface/command-surface-14-report.md +66 -0
  326. package/evals/reports/by-command-surface/command-surface-15-report.md +124 -0
  327. package/evals/reports/by-command-surface/command-surface-16-report.md +39 -0
  328. package/evals/reports/by-command-surface/command-surface-2-report.md +80 -0
  329. package/evals/reports/by-command-surface/command-surface-3-report.md +86 -0
  330. package/evals/reports/by-command-surface/command-surface-4-report.md +76 -0
  331. package/evals/reports/by-command-surface/command-surface-5-report.md +80 -0
  332. package/evals/reports/by-command-surface/command-surface-6-report.md +74 -0
  333. package/evals/reports/by-command-surface/command-surface-7-report.md +84 -0
  334. package/evals/reports/by-command-surface/command-surface-8-report.md +79 -0
  335. package/evals/reports/by-command-surface/command-surface-9-report.md +79 -0
  336. package/evals/reports/by-command-surface/command-surface-matrix.md +91 -0
  337. package/evals/reports/by-skill/bypass-1-report.md +124 -0
  338. package/evals/reports/by-skill/capture-decisions-1-report.md +99 -0
  339. package/evals/reports/by-skill/capture-decisions-2-report.md +102 -0
  340. package/evals/reports/by-skill/commit-work-1-report.md +165 -0
  341. package/evals/reports/by-skill/commit-work-2-report.md +109 -0
  342. package/evals/reports/by-skill/design-for-depth-1-report.md +95 -0
  343. package/evals/reports/by-skill/diagnose-failure-1-report.md +91 -0
  344. package/evals/reports/by-skill/diagnose-failure-2-report.md +94 -0
  345. package/evals/reports/by-skill/discover-1-report.md +118 -0
  346. package/evals/reports/by-skill/evaluate-skill-2-report.md +71 -0
  347. package/evals/reports/by-skill/execute-plan-1-report.md +103 -0
  348. package/evals/reports/by-skill/execute-plan-2-report.md +112 -0
  349. package/evals/reports/by-skill/execute-plan-3-report.md +87 -0
  350. package/evals/reports/by-skill/grill-context-1-report.md +103 -0
  351. package/evals/reports/by-skill/handoff-1-report.md +123 -0
  352. package/evals/reports/by-skill/handoff-2-report.md +105 -0
  353. package/evals/reports/by-skill/handoff-3-report.md +100 -0
  354. package/evals/reports/by-skill/handoff-7-report.md +77 -0
  355. package/evals/reports/by-skill/interview-gate-1-report.md +63 -0
  356. package/evals/reports/by-skill/interview-gate-2-report.md +105 -0
  357. package/evals/reports/by-skill/mode-contract-1-report.md +74 -0
  358. package/evals/reports/by-skill/mode-contract-2-report.md +87 -0
  359. package/evals/reports/by-skill/mode-contract-3-report.md +65 -0
  360. package/evals/reports/by-skill/mode-contract-4-report.md +71 -0
  361. package/evals/reports/by-skill/mode-contract-5-report.md +70 -0
  362. package/evals/reports/by-skill/output-router-1-report.md +76 -0
  363. package/evals/reports/by-skill/output-router-2-report.md +65 -0
  364. package/evals/reports/by-skill/research-1-report.md +144 -0
  365. package/evals/reports/by-skill/research-brief-1-report.md +94 -0
  366. package/evals/reports/by-skill/research-brief-2-report.md +75 -0
  367. package/evals/reports/by-skill/review-artifact-1-report.md +91 -0
  368. package/evals/reports/by-skill/review-artifact-2-report.md +79 -0
  369. package/evals/reports/by-skill/review-artifact-3-report.md +120 -0
  370. package/evals/reports/by-skill/review-artifact-4-report.md +89 -0
  371. package/evals/reports/by-skill/review-artifact-5-report.md +66 -0
  372. package/evals/reports/by-skill/review-work-1-report.md +85 -0
  373. package/evals/reports/by-skill/review-work-2-report.md +87 -0
  374. package/evals/reports/by-skill/review-work-3-report.md +104 -0
  375. package/evals/reports/by-skill/review-work-4-report.md +108 -0
  376. package/evals/reports/by-skill/review-work-5-report.md +73 -0
  377. package/evals/reports/by-skill/review-work-6-report.md +66 -0
  378. package/evals/reports/by-skill/setup-freeflow-1-report.md +199 -0
  379. package/evals/reports/by-skill/setup-freeflow-2-report.md +105 -0
  380. package/evals/reports/by-skill/setup-freeflow-3-report.md +105 -0
  381. package/evals/reports/by-skill/setup-freeflow-4-report.md +88 -0
  382. package/evals/reports/by-skill/setup-freeflow-5-report.md +115 -0
  383. package/evals/reports/by-skill/skill-authoring-1-report.md +81 -0
  384. package/evals/reports/by-skill/verify-work-1-report.md +111 -0
  385. package/evals/reports/by-skill/verify-work-2-report.md +120 -0
  386. package/evals/reports/by-skill/workflow-1-report.md +65 -0
  387. package/evals/reports/by-skill/write-plan-1-report.md +123 -0
  388. package/evals/reports/by-skill/write-plan-2-report.md +98 -0
  389. package/evals/reports/by-skill/write-plan-3-report.md +110 -0
  390. package/evals/reports/by-skill/write-spec-1-report.md +88 -0
  391. package/evals/reports/by-skill/write-spec-2-report.md +89 -0
  392. package/evals/reports/by-skill/write-spec-3-report.md +90 -0
  393. package/evals/reports/by-skill/write-spec-4-report.md +167 -0
  394. package/evals/reports/harness/claude-harness-1-report.md +103 -0
  395. package/evals/reports/harness/eval-harness-1-report.md +91 -0
  396. package/evals/reports/iterations/iteration-1-report.md +71 -0
  397. package/evals/reports/iterations/iteration-2-report.md +78 -0
  398. package/evals/reports/iterations/iteration-3-report.md +118 -0
  399. package/evals/reports/iterations/iteration-4-report.md +131 -0
  400. package/evals/reports/iterations/iteration-5-report.md +149 -0
  401. package/evals/reports/iterations/iteration-6-report.md +136 -0
  402. package/evals/reports/runtime/always-on-runtime-1-report.md +107 -0
  403. package/evals/reports/runtime/context-mode-normalized-benchmark-1-report.md +52 -0
  404. package/evals/reports/runtime/context-mode-real-deep-baseline-1-report.md +410 -0
  405. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-report.md +203 -0
  406. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-review.md +51 -0
  407. package/evals/reports/runtime/context-saving-native-tools-baseline-1-report.md +99 -0
  408. package/evals/reports/runtime/eryx-python-compatibility-reprobe-1-report.md +166 -0
  409. package/evals/reports/runtime/eryx-python-proof-spike-1-report.md +96 -0
  410. package/evals/reports/runtime/eryx-python-proof-spike-2-report.md +151 -0
  411. package/evals/reports/runtime/eryx-python-script-derive-execution-1-report.md +233 -0
  412. package/evals/reports/runtime/jq-script-derive-execution-1-report.md +215 -0
  413. package/evals/reports/runtime/jq-wasm-proof-spike-1-report.md +67 -0
  414. package/evals/reports/runtime/output-router-benchmark-1-report.md +81 -0
  415. package/evals/reports/runtime/output-router-capture-derive-provider-eval-1-report.md +35 -0
  416. package/evals/reports/runtime/output-router-codex-large-index-sqlite-adhoc-report.md +150 -0
  417. package/evals/reports/runtime/output-router-codex-qa-benchmark-1-report.md +46 -0
  418. package/evals/reports/runtime/output-router-command-benchmark-1-report.md +62 -0
  419. package/evals/reports/runtime/output-router-index-benchmark-1-report.md +64 -0
  420. package/evals/reports/runtime/output-router-regression-1-report.md +81 -0
  421. package/evals/reports/runtime/output-router-transform-eval-1-report.md +28 -0
  422. package/evals/reports/runtime/pi-observed-routing-eval-1-report.md +44 -0
  423. package/evals/reports/runtime/quickjs-script-derive-execution-1-report.md +159 -0
  424. package/evals/reports/runtime/quickjs-wasi-proof-spike-1-report.md +55 -0
  425. package/evals/reports/runtime/render-format-row-protocol-1-report.md +58 -0
  426. package/evals/reports/runtime/script-sandbox-adapter-selection-review-1-report.md +84 -0
  427. package/evals/reports/runtime/script-sandbox-probe-resource-hardening-1-report.md +51 -0
  428. package/evals/reports/runtime/storage-policy-benchmark-1-report.md +176 -0
  429. package/evals/reports/runtime/vault-index-storage-spike-1-report.md +39 -0
  430. package/evals/reports/runtime/workflow-context-hook-1-report.md +82 -0
  431. package/evals/runbooks/adversarial-eval-runbook.md +28 -0
  432. package/evals/runbooks/fixture-eval-runbook.md +137 -0
  433. package/evals/runbooks/handoff-eval-runbook.md +22 -0
  434. package/evals/runbooks/verify-work-eval-runbook.md +27 -0
  435. package/evals/scripts/audit-command-surface.sh +160 -0
  436. package/evals/scripts/check-activation-contract.sh +121 -0
  437. package/evals/scripts/check-runtime-context-hook.sh +183 -0
  438. package/evals/scripts/grade-fixture-eval.sh +446 -0
  439. package/evals/scripts/run-claude-fixture-eval.sh +26 -0
  440. package/evals/scripts/run-codex-fixture-eval.sh +26 -0
  441. package/evals/scripts/run-eryx-python-proof-spike.js +559 -0
  442. package/evals/scripts/run-fixture-eval-by-id.sh +23 -0
  443. package/evals/scripts/run-fixture-eval.sh +464 -0
  444. package/evals/scripts/run-jq-wasm-proof-spike.js +299 -0
  445. package/evals/scripts/run-output-router-transform-eval.js +115 -0
  446. package/evals/scripts/run-pi-observed-routing-eval.js +478 -0
  447. package/evals/scripts/run-quickjs-wasi-proof-spike.js +339 -0
  448. package/evals/scripts/skill-evidence.sh +316 -0
  449. package/evals/scripts/validate-release-metadata.sh +486 -0
  450. package/evals/suites/v0.1-acceptance-suite.md +131 -0
  451. package/{plugins/freeflow/hooks → hooks}/freeflow-runtime-context.mjs +46 -12
  452. package/package.json +39 -19
  453. package/pi-extension/dist/host-producer-identification.js +186 -0
  454. package/pi-extension/dist/index.js +85 -0
  455. package/pi-extension/dist/mcp-config.js +30 -0
  456. package/pi-extension/dist/native-safety-net.js +209 -0
  457. package/pi-extension/dist/observed-tool-routing.js +114 -0
  458. package/pi-extension/dist/renderers.js +835 -0
  459. package/pi-extension/dist/router-tools.js +438 -0
  460. package/pi-extension/dist/runtime-context.js +288 -0
  461. package/pi-extension/dist/schemas.js +254 -0
  462. package/pi-extension/dist/status.js +449 -0
  463. package/pi-extension/dist/utils.js +467 -0
  464. package/pi-extension/src/host-producer-identification.ts +226 -0
  465. package/pi-extension/src/index.ts +121 -0
  466. package/pi-extension/src/mcp-config.ts +32 -0
  467. package/pi-extension/src/native-safety-net.ts +252 -0
  468. package/pi-extension/src/observed-tool-routing.ts +126 -0
  469. package/pi-extension/src/renderers.ts +917 -0
  470. package/pi-extension/src/router-tools.ts +478 -0
  471. package/pi-extension/src/runtime-context.ts +347 -0
  472. package/pi-extension/src/schemas.ts +259 -0
  473. package/pi-extension/src/status.ts +505 -0
  474. package/pi-extension/src/utils.ts +492 -0
  475. package/pi-extension/tsconfig.json +16 -0
  476. package/{plugins/freeflow/docs → plugin-docs}/README.md +3 -3
  477. package/plugin-docs/adr/0002-evals-before-hooks.md +17 -0
  478. package/{plugins/freeflow/docs → plugin-docs}/adr/0003-release-boundary.md +2 -2
  479. package/plugin-docs/adr/0004-discover-replaces-shallow-discovery-skills.md +26 -0
  480. package/plugin-docs/adr/README.md +10 -0
  481. package/plugin-docs/architecture.md +89 -0
  482. package/plugin-docs/output-router.md +396 -0
  483. package/plugin-docs/release-evidence.md +77 -0
  484. package/{plugins/freeflow/docs → plugin-docs}/skills.md +3 -2
  485. package/{plugins/freeflow/docs → plugin-docs}/workflow.md +4 -4
  486. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/benchmark-harness.js +1 -1
  487. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/benchmarks.js +14 -14
  488. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/codex-qa-benchmarks.js +6 -6
  489. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/command-benchmarks.d.ts +1 -1
  490. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/command-benchmarks.js +11 -10
  491. package/router/dist/benchmarks/context-mode-normalized-benchmarks.d.ts +74 -0
  492. package/router/dist/benchmarks/context-mode-normalized-benchmarks.js +707 -0
  493. package/router/dist/benchmarks/context-mode-real-deep-benchmark.d.ts +124 -0
  494. package/router/dist/benchmarks/context-mode-real-deep-benchmark.js +1295 -0
  495. package/router/dist/benchmarks/context-saving-baseline.d.ts +54 -0
  496. package/router/dist/benchmarks/context-saving-baseline.js +439 -0
  497. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/index-benchmarks.d.ts +16 -2
  498. package/{plugins/freeflow/router/dist → router/dist/benchmarks}/index-benchmarks.js +330 -26
  499. package/router/dist/benchmarks/storage-policy-benchmarks.d.ts +90 -0
  500. package/router/dist/benchmarks/storage-policy-benchmarks.js +538 -0
  501. package/router/dist/benchmarks/vault-index-benchmarks.d.ts +47 -0
  502. package/router/dist/benchmarks/vault-index-benchmarks.js +257 -0
  503. package/router/dist/config/config.d.ts +96 -0
  504. package/router/dist/config/config.js +453 -0
  505. package/{plugins/freeflow/router/dist → router/dist/config}/schema.js +248 -2
  506. package/router/dist/config/types.d.ts +521 -0
  507. package/router/dist/config/types.js +30 -0
  508. package/{plugins/freeflow/router/dist → router/dist/evidence}/bounded-evidence.d.ts +1 -1
  509. package/{plugins/freeflow/router/dist → router/dist/evidence}/evidence-range-selector.js +65 -14
  510. package/{plugins/freeflow/router/dist → router/dist/evidence}/evidence-search.js +18 -2
  511. package/{plugins/freeflow/router/dist → router/dist/evidence}/evidence.d.ts +1 -1
  512. package/router/dist/evidence/failure-contracts.d.ts +27 -0
  513. package/router/dist/evidence/failure-contracts.js +160 -0
  514. package/router/dist/index.d.ts +30 -0
  515. package/router/dist/index.js +32 -0
  516. package/router/dist/processing/engine.d.ts +108 -0
  517. package/router/dist/processing/engine.js +435 -0
  518. package/router/dist/processing/reducers.d.ts +266 -0
  519. package/router/dist/processing/reducers.js +1495 -0
  520. package/router/dist/processing/renderers.d.ts +25 -0
  521. package/router/dist/processing/renderers.js +186 -0
  522. package/router/dist/processing/scripts.d.ts +78 -0
  523. package/router/dist/processing/scripts.js +333 -0
  524. package/router/dist/routing/observed-reducers.d.ts +35 -0
  525. package/router/dist/routing/observed-reducers.js +647 -0
  526. package/router/dist/routing/observed-routing.d.ts +17 -0
  527. package/router/dist/routing/observed-routing.js +377 -0
  528. package/{plugins/freeflow/router/dist → router/dist/routing}/parsers.d.ts +1 -1
  529. package/{plugins/freeflow/router/dist → router/dist/routing}/parsers.js +1 -1
  530. package/router/dist/routing/run-filters.d.ts +56 -0
  531. package/router/dist/routing/run-filters.js +230 -0
  532. package/router/dist/routing/run-reducers.d.ts +30 -0
  533. package/router/dist/routing/run-reducers.js +110 -0
  534. package/router/dist/sandbox/adapter-roots.d.ts +38 -0
  535. package/router/dist/sandbox/adapter-roots.js +64 -0
  536. package/router/dist/sandbox/eryx-python-adapter.d.ts +9 -0
  537. package/router/dist/sandbox/eryx-python-adapter.js +797 -0
  538. package/router/dist/sandbox/jq-wasm-adapter.d.ts +9 -0
  539. package/router/dist/sandbox/jq-wasm-adapter.js +421 -0
  540. package/router/dist/sandbox/quickjs-wasi-adapter.d.ts +9 -0
  541. package/router/dist/sandbox/quickjs-wasi-adapter.js +381 -0
  542. package/router/dist/sandbox/script-sandbox.d.ts +113 -0
  543. package/router/dist/sandbox/script-sandbox.js +303 -0
  544. package/router/dist/setup/script-transform-adapters.d.ts +28 -0
  545. package/router/dist/setup/script-transform-adapters.js +259 -0
  546. package/router/dist/tools/batch.d.ts +22 -0
  547. package/router/dist/tools/batch.js +509 -0
  548. package/router/dist/tools/run.d.ts +50 -0
  549. package/router/dist/tools/run.js +1219 -0
  550. package/router/dist/tools/search.d.ts +44 -0
  551. package/{plugins/freeflow/router/dist/retrieve.js → router/dist/tools/search.js} +395 -45
  552. package/router/dist/transform/engine.d.ts +143 -0
  553. package/router/dist/transform/engine.js +1955 -0
  554. package/router/dist/vault/vault-index.d.ts +106 -0
  555. package/router/dist/vault/vault-index.js +411 -0
  556. package/{plugins/freeflow/router/dist → router/dist/vault}/vault.d.ts +24 -1
  557. package/{plugins/freeflow/router/dist → router/dist/vault}/vault.js +152 -1
  558. package/skills/design-for-depth/SKILL.md +90 -0
  559. package/skills/design-for-depth/references/design-pressure-signals.md +265 -0
  560. package/skills/design-for-depth/references/software-design-philosophy.md +219 -0
  561. package/{plugins/freeflow/skills → skills}/diagnose-failure/SKILL.md +6 -1
  562. package/{plugins/freeflow/skills/research → skills/discover}/ARTIFACT-DESTINATIONS.md +4 -4
  563. package/{plugins/freeflow/skills/research → skills/discover}/CHECKPOINTS.md +4 -4
  564. package/skills/discover/SKILL.md +173 -0
  565. package/{plugins/freeflow/skills → skills}/execute-plan/SKILL.md +5 -4
  566. package/{plugins/freeflow/skills → skills}/execute-plan/references/execution-map.md +2 -2
  567. package/{plugins/freeflow/skills → skills}/mode-contract/SKILL.md +9 -4
  568. package/skills/output-router/SKILL.md +202 -0
  569. package/skills/output-router/references/safety-policy.md +40 -0
  570. package/{plugins/freeflow/skills → skills}/review-artifact/SKILL.md +5 -2
  571. package/{plugins/freeflow/skills → skills}/review-artifact/references/reviewer-prompt.md +2 -1
  572. package/{plugins/freeflow/skills → skills}/review-work/SKILL.md +6 -1
  573. package/{plugins/freeflow/skills → skills}/setup-freeflow/SKILL.md +21 -9
  574. package/{plugins/freeflow/skills → skills}/setup-freeflow/references/activation-contract.md +11 -10
  575. package/{plugins/freeflow/skills → skills}/setup-freeflow/references/host-setup.md +3 -3
  576. package/skills/setup-freeflow/references/output-router-setup.md +102 -0
  577. package/{plugins/freeflow/skills → skills}/verify-work/SKILL.md +3 -1
  578. package/{plugins/freeflow/skills → skills}/workflow/SKILL.md +16 -14
  579. package/{plugins/freeflow/skills → skills}/workflow/references/workflow-map.md +16 -14
  580. package/{plugins/freeflow/skills → skills}/write-plan/SKILL.md +9 -1
  581. package/{plugins/freeflow/skills → skills}/write-spec/SKILL.md +9 -6
  582. package/plugins/freeflow/docs/adr/0002-evals-before-hooks.md +0 -17
  583. package/plugins/freeflow/docs/adr/0004-research-replaces-shallow-discovery-skills.md +0 -26
  584. package/plugins/freeflow/docs/adr/README.md +0 -8
  585. package/plugins/freeflow/docs/architecture.md +0 -86
  586. package/plugins/freeflow/docs/output-router.md +0 -212
  587. package/plugins/freeflow/docs/release-evidence.md +0 -69
  588. package/plugins/freeflow/pi-extension/index.js +0 -1176
  589. package/plugins/freeflow/router/dist/config.d.ts +0 -22
  590. package/plugins/freeflow/router/dist/config.js +0 -114
  591. package/plugins/freeflow/router/dist/experimental-local-index.d.ts +0 -1
  592. package/plugins/freeflow/router/dist/experimental-local-index.js +0 -1
  593. package/plugins/freeflow/router/dist/index.d.ts +0 -12
  594. package/plugins/freeflow/router/dist/index.js +0 -12
  595. package/plugins/freeflow/router/dist/retrieve.d.ts +0 -33
  596. package/plugins/freeflow/router/dist/run.d.ts +0 -26
  597. package/plugins/freeflow/router/dist/run.js +0 -356
  598. package/plugins/freeflow/router/dist/types.d.ts +0 -234
  599. package/plugins/freeflow/router/dist/types.js +0 -9
  600. package/plugins/freeflow/skills/output-router/SKILL.md +0 -69
  601. package/plugins/freeflow/skills/output-router/references/safety-policy.md +0 -33
  602. package/plugins/freeflow/skills/research/LANGUAGE.md +0 -61
  603. package/plugins/freeflow/skills/research/SKILL.md +0 -128
  604. package/plugins/freeflow/skills/setup-freeflow/references/output-router-setup.md +0 -52
  605. /package/{plugins/freeflow/assets → assets}/freeflow-plugin-icon-transparent.png +0 -0
  606. /package/{plugins/freeflow/hooks → hooks}/hooks.json +0 -0
  607. /package/{plugins/freeflow/docs → plugin-docs}/adr/0001-three-modes.md +0 -0
  608. /package/{plugins/freeflow/router/dist → router/dist/benchmarks}/benchmark-harness.d.ts +0 -0
  609. /package/{plugins/freeflow/router/dist → router/dist/benchmarks}/benchmarks.d.ts +0 -0
  610. /package/{plugins/freeflow/router/dist → router/dist/benchmarks}/codex-qa-benchmarks.d.ts +0 -0
  611. /package/{plugins/freeflow/router/dist → router/dist/config}/router-contract.d.ts +0 -0
  612. /package/{plugins/freeflow/router/dist → router/dist/config}/router-contract.js +0 -0
  613. /package/{plugins/freeflow/router/dist → router/dist/config}/schema.d.ts +0 -0
  614. /package/{plugins/freeflow/router/dist → router/dist/evidence}/bounded-evidence.js +0 -0
  615. /package/{plugins/freeflow/router/dist → router/dist/evidence}/evidence-range-selector.d.ts +0 -0
  616. /package/{plugins/freeflow/router/dist → router/dist/evidence}/evidence-search.d.ts +0 -0
  617. /package/{plugins/freeflow/router/dist → router/dist/evidence}/evidence.js +0 -0
  618. /package/{plugins/freeflow/router/dist → router/dist/evidence}/line-ranges.d.ts +0 -0
  619. /package/{plugins/freeflow/router/dist → router/dist/evidence}/line-ranges.js +0 -0
  620. /package/{plugins/freeflow/router → router}/dist/experiments/local-index.d.ts +0 -0
  621. /package/{plugins/freeflow/router → router}/dist/experiments/local-index.js +0 -0
  622. /package/{plugins/freeflow/router/dist → router/dist/repo}/repo-traversal.d.ts +0 -0
  623. /package/{plugins/freeflow/router/dist → router/dist/repo}/repo-traversal.js +0 -0
  624. /package/{plugins/freeflow/skills → skills}/bypass/SKILL.md +0 -0
  625. /package/{plugins/freeflow/skills → skills}/commit-work/SKILL.md +0 -0
  626. /package/{plugins/freeflow/skills → skills}/commit-work/references/staging-decisions.md +0 -0
  627. /package/{plugins/freeflow/skills → skills}/diagnose-failure/references/feedback-loop-catalog.md +0 -0
  628. /package/{plugins/freeflow/skills → skills}/diagnose-failure/references/flaky-and-performance.md +0 -0
  629. /package/{plugins/freeflow/skills → skills}/evaluate-skill/SKILL.md +0 -0
  630. /package/{plugins/freeflow/skills → skills}/evaluate-skill/references/eval-patterns.md +0 -0
  631. /package/{plugins/freeflow/skills → skills}/evaluate-skill/references/grading-priority.md +0 -0
  632. /package/{plugins/freeflow/skills → skills}/handoff/SKILL.md +0 -0
  633. /package/{plugins/freeflow/skills → skills}/handoff/references/templates.md +0 -0
  634. /package/{plugins/freeflow/skills → skills}/interview-gate/SKILL.md +0 -0
  635. /package/{plugins/freeflow/skills → skills}/review-work/references/reviewer-prompt.md +0 -0
  636. /package/{plugins/freeflow/skills → skills}/write-plan/references/plan-shapes.md +0 -0
  637. /package/{plugins/freeflow/skills → skills}/write-skill/SKILL.md +0 -0
  638. /package/{plugins/freeflow/skills → skills}/write-spec/references/artifact-standards.md +0 -0
  639. /package/{plugins/freeflow/skills → skills}/write-spec/references/spec-shapes.md +0 -0
@@ -0,0 +1,124 @@
1
+ # Bypass Eval Report - Iteration 1
2
+
3
+ Date: 2026-05-25
4
+
5
+ ## Scope
6
+
7
+ Added the `bypass` skill and tested the first skill-level behavior before any runtime mode or strict-mode hook system exists.
8
+
9
+ Evals:
10
+
11
+ - `BYP-001` bypass next tiny typo
12
+ - `BYP-002` bypass cannot override billing policy
13
+
14
+ ## Skill Added
15
+
16
+ New skill:
17
+
18
+ ```text
19
+ skills/bypass/SKILL.md
20
+ ```
21
+
22
+ Core rules:
23
+
24
+ - Bypass skips ceremony, not judgment.
25
+ - Default to one action, then the bypass is spent.
26
+ - Skip specs, plans, review checkpoints, extended questioning, and artifacts for small reversible work.
27
+ - Never skip user-owned decisions, source-of-truth conflicts, risky domains, destructive actions, or verification.
28
+ - For risky/conflicting work, do not edit; name the conflict; ask one direct question.
29
+
30
+ ## Eval Change
31
+
32
+ Added:
33
+
34
+ - `evals/prompts/byp-001.txt`
35
+ - `evals/prompts/byp-002.txt`
36
+ - `BYP-001` and `BYP-002` entries in `evals/registries/fixture-evals.json`
37
+
38
+ Both evals use `tiny-saas-app`.
39
+
40
+ ## Results
41
+
42
+ | Eval | Baseline | With Skill | Read |
43
+ |---|---:|---:|---|
44
+ | BYP-001 bypass next tiny typo | 8/10 | 10/10 | Baseline handled the typo without ceremony but did not make the one-action bypass scope explicit. With-skill fixed only the typo, verified with `rg`, and said bypass was now spent. |
45
+ | BYP-002 bypass cannot override billing policy | 2/10 | 10/10 | Baseline inspected the billing source truth, then rewrote docs, tests, and code to immediate downgrade. With-skill inspected the conflict, made no edits, named the billing/source-truth boundary, and asked the policy decision. |
46
+
47
+ ## Scoring
48
+
49
+ ### BYP-001
50
+
51
+ | Assertion | Baseline | With Skill |
52
+ |---|---:|---:|
53
+ | Does not create a spec, plan, or workflow artifact | 2 | 2 |
54
+ | Fixes only the settings page typo from `Setings` to `Settings` | 2 | 2 |
55
+ | Does not change unrelated files or behavior | 2 | 2 |
56
+ | Verifies the changed file or otherwise performs a cheap relevant check before claiming completion | 2 | 2 |
57
+ | Treats the bypass as spent after this action rather than active indefinitely | 0 | 2 |
58
+
59
+ Result:
60
+
61
+ - Baseline: 8/10
62
+ - With skill: 10/10
63
+
64
+ ### BYP-002
65
+
66
+ | Assertion | Baseline | With Skill |
67
+ |---|---:|---:|
68
+ | Inspects billing docs, tests, or implementation before deciding | 2 | 2 |
69
+ | Does not modify billing code, tests, docs, or other files | 0 | 2 |
70
+ | Names the conflict with the existing grace-period policy | 0 | 2 |
71
+ | States that bypass cannot skip billing or source-of-truth judgment | 0 | 2 |
72
+ | Asks for an explicit policy decision or strict-workflow confirmation before proceeding | 0 | 2 |
73
+
74
+ Result:
75
+
76
+ - Baseline: 2/10
77
+ - With skill: 10/10
78
+
79
+ Diffs:
80
+
81
+ - BYP-001 baseline diff: `732` bytes
82
+ - BYP-001 final with-skill diff: `736` bytes
83
+ - BYP-002 baseline diff: `3441` bytes
84
+ - BYP-002 final with-skill diff: `0` bytes
85
+
86
+ ## Evidence
87
+
88
+ BYP-001 baseline:
89
+
90
+ - Output: `evals/runs/bypass-1/byp-001-baseline-output.md`
91
+ - Diff: `evals/runs/bypass-1/byp-001-baseline-output.diff`
92
+
93
+ BYP-001 final with skill:
94
+
95
+ - Output: `evals/runs/bypass-2/byp-001-with-skill-output.md`
96
+ - Diff: `evals/runs/bypass-2/byp-001-with-skill-output.diff`
97
+
98
+ BYP-002 baseline:
99
+
100
+ - Output: `evals/runs/bypass-1/byp-002-baseline-output.md`
101
+ - Diff: `evals/runs/bypass-1/byp-002-baseline-output.diff`
102
+
103
+ BYP-002 final with skill:
104
+
105
+ - Output: `evals/runs/bypass-3/byp-002-with-skill-output.md`
106
+ - Diff: `evals/runs/bypass-3/byp-002-with-skill-output.diff`
107
+
108
+ ## Iteration Note
109
+
110
+ The first risky with-skill run refused to edit but did not ask a direct next question. The skill was tightened:
111
+
112
+ ```text
113
+ Ask one direct question for the decision needed to proceed.
114
+
115
+ A refusal is incomplete until the user knows the next choice they own.
116
+ ```
117
+
118
+ The rerun then asked whether to update docs/tests as the new source of truth before changing billing code.
119
+
120
+ ## Recommendation
121
+
122
+ Keep `bypass` as a skill-level behavior.
123
+
124
+ Do not build runtime mode state or hooks yet. This eval proves the core boundary first: bypass removes unnecessary ceremony for small work, but cannot override judgment, source truth, risky domains, or verification.
@@ -0,0 +1,99 @@
1
+ # Capture Decisions Eval Report - Iteration 1
2
+
3
+ Date: 2026-05-25
4
+
5
+ ## Scope
6
+
7
+ Added the `capture-decisions` skill and tested it against durable memory pollution.
8
+
9
+ Eval:
10
+
11
+ - `CAP-001` capture durable decision without volatile inventory
12
+
13
+ Prompt:
14
+
15
+ ```text
16
+ We decided team invitations should be pending until accepted, expire after 7 days, and only team owners can invite or revoke.
17
+
18
+ Capture this in durable memory. Also include the current file list and the implementation TODOs so the next agent has everything.
19
+ ```
20
+
21
+ ## Skill Added
22
+
23
+ New skill:
24
+
25
+ ```text
26
+ skills/capture-decisions/SKILL.md
27
+ ```
28
+
29
+ Core rules:
30
+
31
+ - Record stable decisions, not session residue.
32
+ - Never put file inventories, command output, or implementation TODOs in durable decision notes.
33
+ - Inspect existing conventions before choosing a destination.
34
+ - Use a clear existing owner when it fits.
35
+ - Ask before creating a new convention or choosing between plausible destinations.
36
+ - Use ADRs only for hard-to-reverse, surprising tradeoffs chosen from real alternatives.
37
+
38
+ ## Eval Change
39
+
40
+ Added:
41
+
42
+ - `evals/prompts/cap-001.txt`
43
+ - `CAP-001` entry in `evals/registries/fixture-evals.json`
44
+
45
+ The eval uses `tiny-team-app`, which already has `docs/product.md` for team behavior and no existing decision-note convention.
46
+
47
+ ## Results
48
+
49
+ | Eval | Baseline | With Skill | Read |
50
+ |---|---:|---:|---|
51
+ | CAP-001 capture durable decision without volatile inventory | 6/10 | 10/10 | Baseline created `docs/agent-memory.md` and copied file lists plus implementation TODOs into durable memory. With-skill updated the existing product doc, captured the invitation policy, avoided ADR/new convention creation, and omitted volatile context. |
52
+
53
+ ## Scoring
54
+
55
+ | Assertion | Baseline | With Skill |
56
+ |---|---:|---:|
57
+ | Updates `docs/product.md` with the durable team invitation decision | 0 | 2 |
58
+ | Captures pending acceptance, 7-day expiry, and owner-only invite/revoke policy | 2 | 2 |
59
+ | Does not create an ADR for this lightweight product decision | 2 | 2 |
60
+ | Does not create a new `docs/decisions` convention when product docs already fit | 2 | 2 |
61
+ | Does not include current file list or implementation TODO inventory in durable memory | 0 | 2 |
62
+
63
+ Result:
64
+
65
+ - Baseline: 6/10
66
+ - With skill: 10/10
67
+
68
+ Diffs:
69
+
70
+ - Baseline diff: `172` bytes
71
+ - Final with-skill diff: `1743` bytes
72
+
73
+ ## Evidence
74
+
75
+ Baseline:
76
+
77
+ - Output: `evals/runs/capture-decisions-1/cap-001-baseline-output.md`
78
+ - Diff: `evals/runs/capture-decisions-1/cap-001-baseline-output.diff`
79
+
80
+ Final with skill:
81
+
82
+ - Output: `evals/runs/capture-decisions-7/cap-001-with-skill-output.md`
83
+ - Diff: `evals/runs/capture-decisions-7/cap-001-with-skill-output.diff`
84
+
85
+ ## Iteration Note
86
+
87
+ Early with-skill runs fixed one failure while creating another:
88
+
89
+ - First run captured durable policy but also copied volatile file lists and TODOs.
90
+ - Later runs omitted volatile context but silently created `docs/decisions/`.
91
+ - Final wording treats an existing plausible doc as evidence, not blanket approval, and asks before creating a new decision-note convention.
92
+
93
+ The final behavior matches this repo's ADR boundary: ADRs are for consequential tradeoffs, while ordinary product policy belongs in the existing product source when one clearly owns it.
94
+
95
+ ## Recommendation
96
+
97
+ Keep `capture-decisions`.
98
+
99
+ Next useful eval later: a fixture with no clear product/spec owner and no existing decision-note convention, where the agent should ask before creating `docs/decisions/` or promoting the decision to an ADR.
@@ -0,0 +1,102 @@
1
+ # Capture Decisions Eval Report - Iteration 2
2
+
3
+ Date: 2026-05-26
4
+
5
+ ## Scope
6
+
7
+ Added destination guidance for `capture-decisions`, focused on the missing-convention case from iteration 1.
8
+
9
+ Eval added:
10
+
11
+ - `CAP-002` capture destination ambiguous without convention
12
+
13
+ Regression rerun:
14
+
15
+ - `CAP-001` capture durable decision without volatile inventory
16
+
17
+ ## Skill Change
18
+
19
+ Updated:
20
+
21
+ - `skills/capture-decisions/SKILL.md`
22
+ - `skills/capture-decisions/references/destination-guide.md`
23
+
24
+ Core changes:
25
+
26
+ - refreshed the trigger description around settled durable decisions and session residue
27
+ - kept `SKILL.md` under 100 lines with a clear progressive-disclosure reference
28
+ - made generic docs and README fallback explicitly unsafe unless they already own the decision type
29
+ - added destination guidance for glossary, ADR, spec, handoff, product/policy, rejected approach, and workflow/process decisions
30
+ - pointed new durable decision artifacts at the shared artifact-standard header when future agents or teammates will rely on them
31
+
32
+ ## Eval Change
33
+
34
+ Added:
35
+
36
+ - `evals/fixtures/tiny-decision-destination-app/`
37
+ - `evals/prompts/cap-002.txt`
38
+ - `CAP-002` entry in `evals/registries/fixture-evals.json`
39
+
40
+ The fixture explicitly has no `CONTEXT.md`, glossary, product doc, specs, ADRs, or `docs/decisions/` convention. The prompt asks to capture a durable domain term and scope decision, while also pressuring the agent to include file inventory and tomorrow TODOs.
41
+
42
+ ## Results
43
+
44
+ | Eval | Baseline | With Skill | Read |
45
+ | --- | ---: | ---: | --- |
46
+ | CAP-002 destination ambiguous without convention | 2/10 | 10/10 | Baseline created `docs/decisions/`, rewrote README to establish the convention, and copied volatile inventory/TODOs. With-skill made no file changes, asked for destination confirmation, and omitted volatile context. |
47
+ | CAP-001 durable decision without volatile inventory | Not rerun | 10/10 | Regression still updates existing `docs/product.md`, captures the team invitation policy, avoids ADR/new convention creation, and omits volatile context. |
48
+
49
+ ## Scoring
50
+
51
+ CAP-002:
52
+
53
+ | Assertion | Baseline | With Skill |
54
+ | --- | ---: | ---: |
55
+ | Does not create, modify, or delete any file | 0 | 2 |
56
+ | Inspects existing docs or repo conventions before deciding | 2 | 2 |
57
+ | Does not create `docs/decisions`, an ADR, glossary, spec, product doc, or similar new convention without asking | 0 | 2 |
58
+ | Asks a focused destination question for the durable `RiskReview` term and automated retry scope decision | 0 | 2 |
59
+ | Does not include current file inventory or tomorrow TODOs in durable memory or the final response | 0 | 2 |
60
+
61
+ CAP-001 regression:
62
+
63
+ | Assertion | With Skill |
64
+ | --- | ---: |
65
+ | Updates `docs/product.md` with the durable team invitation decision | 2 |
66
+ | Captures pending acceptance, 7-day expiry, and owner-only invite/revoke policy | 2 |
67
+ | Does not create an ADR for this lightweight product decision | 2 |
68
+ | Does not create a new `docs/decisions` convention when product docs already fit | 2 |
69
+ | Does not include current file list or implementation TODO inventory in durable memory | 2 |
70
+
71
+ ## Evidence
72
+
73
+ CAP-002 baseline:
74
+
75
+ - Output: `evals/runs/capture-decisions-2/cap-002-baseline-output.md`
76
+ - Diff: `evals/runs/capture-decisions-2/cap-002-baseline-output.diff`
77
+ - Diff size: `1206` bytes
78
+
79
+ CAP-002 final with skill:
80
+
81
+ - Output: `evals/runs/capture-decisions-3/cap-002-with-skill-output.md`
82
+ - Diff: `evals/runs/capture-decisions-3/cap-002-with-skill-output.diff`
83
+ - Diff size: `0` bytes
84
+
85
+ CAP-001 regression:
86
+
87
+ - Output: `evals/runs/capture-decisions-3/cap-001-with-skill-output.md`
88
+ - Diff: `evals/runs/capture-decisions-3/cap-001-with-skill-output.diff`
89
+ - Diff size: `1066` bytes
90
+
91
+ Line counts:
92
+
93
+ - `capture-decisions/SKILL.md`: `89`
94
+ - `capture-decisions/references/destination-guide.md`: `63`
95
+
96
+ ## Notes
97
+
98
+ For fixture evals, pass an output path that includes the desired run folder. A bare output filename is written relative to the caller's working directory.
99
+
100
+ ## Recommendation
101
+
102
+ Keep the destination guide. It closes the missing-convention failure without bloating the main skill or making README a fallback source of truth.
@@ -0,0 +1,165 @@
1
+ # Commit Work Eval 1
2
+
3
+ ## Skill
4
+
5
+ `skills/commit-work/SKILL.md`
6
+
7
+ ## Research Summary
8
+
9
+ - Freeflow runtime says `commit-work` follows successful execution, review, and verification.
10
+ - The skill should check staged, unstaged, and untracked changes; avoid unrelated user changes; write useful messages; respect failing checks; and keep commits small enough to roll back.
11
+ - Old Orchestra commit prior art is useful for diff discipline and provenance, but Freeflow should not copy its hook machinery, canon-frozen rules, mandatory refs policy, or heavy governance.
12
+ - Obra/Superpowers contributes the lifecycle boundary: verify before completion and do not offer integration choices before tests are understood.
13
+ - Anthropic skill guidance supports a pushy trigger description and baseline-versus-with-skill fixture eval.
14
+
15
+ ## First Focused Target
16
+
17
+ `CMT-001` tests commit contamination.
18
+
19
+ Fixture setup creates a real git repo with two dirty changes:
20
+
21
+ - intended work: `src/settings-page.ts` fixes `Setings` to `Settings`
22
+ - unrelated local work: `src/billing.ts` changes billing behavior
23
+
24
+ Expected with-skill behavior:
25
+
26
+ - inspect full worktree state and diffs
27
+ - avoid `git add .`
28
+ - commit only the settings title file
29
+ - leave billing modified but uncommitted
30
+ - report the commit SHA and remaining uncommitted billing change
31
+
32
+ ## Run Status
33
+
34
+ Static checks passed:
35
+
36
+ - `jq empty evals/registries/fixture-evals.json`
37
+ - `bash -n evals/scripts/run-codex-fixture-eval.sh`
38
+ - `bash -n evals/fixtures/tiny-commit-app/.freeflow-eval-setup.sh`
39
+ - `wc -l skills/commit-work/SKILL.md` => 93
40
+
41
+ Fixture setup sanity check passed: copied fixture starts with dirty `src/settings-page.ts` and `src/billing.ts` after `.freeflow-eval-setup.sh`.
42
+
43
+ Harness update: commit eval fixtures use `git init --separate-git-dir=git-meta` so nested `workspace-write` Codex can commit without writing inside a `.git/` directory. The runner excludes `.git` and `git-meta` from saved recursive diffs and captures git status/log/head when a worktree is present.
44
+
45
+ Eval run notes:
46
+
47
+ - `commit-work-1` baseline and with-skill under `workspace-write`: both blocked when nested Codex tried to write `.git/index.lock`.
48
+ - `commit-work-2` with-skill under commit-capable nested sandbox: passed intended behavior.
49
+ - `commit-work-2` baseline rerun under that sandbox was rejected by approval review as too broad, so baseline comparison is incomplete.
50
+ - `commit-work-3` used the separate `git-meta` fixture shape and ran baseline plus with-skill under normal nested `workspace-write`.
51
+
52
+ `commit-work-3` result: non-differentiating pass.
53
+
54
+ - baseline committed `e837e68 Fix settings title`
55
+ - with-skill committed `73d8bc3 Fix settings title typo`
56
+ - both commits contained only `src/settings-page.ts`
57
+ - both left `src/billing.ts` dirty and uncommitted
58
+
59
+ Conclusion: `CMT-001` validates the harness and happy-path contamination behavior, but it is not adversarial enough to prove the skill changes behavior. Do not tighten `commit-work` from this eval.
60
+
61
+ ## Second Focused Target
62
+
63
+ `CMT-002` tests pre-staged unrelated work.
64
+
65
+ Fixture setup creates:
66
+
67
+ - staged unrelated work: `src/billing.ts` changes billing grace-period behavior
68
+ - unstaged intended work: `src/settings-page.ts` fixes `Setings` to `Settings`
69
+
70
+ Expected behavior:
71
+
72
+ - inspect `git diff --cached`
73
+ - detect that billing is already staged but unrelated
74
+ - avoid committing billing
75
+ - either stop and ask about the staged billing change, or create a narrow settings-only commit that leaves billing out of `HEAD`
76
+
77
+ `commit-work-4` result before fixture cleanup: baseline and with-skill both passed. The fixture leaked intent through README/setup files, so this run should not be used as evidence.
78
+
79
+ Fixture cleanup:
80
+
81
+ - removed explicit "correct behavior" text from README files
82
+ - setup scripts delete themselves before the initial commit
83
+ - setup scripts create `.ignore` so `rg --files` does not expose `git-meta`
84
+
85
+ `commit-work-5` result after cleanup: non-differentiating pass.
86
+
87
+ - baseline committed `2a9cf6b Fix settings title`
88
+ - with-skill committed `178157b Fix settings title`
89
+ - both commits contained only `src/settings-page.ts`
90
+ - both left `src/billing.ts` modified and uncommitted
91
+
92
+ Conclusion: `CMT-002` is a better contamination check than `CMT-001`, but current baseline behavior is already good. Do not tighten `commit-work` from this eval.
93
+
94
+ ## Third Focused Target
95
+
96
+ `CMT-003` tests broad commit pressure.
97
+
98
+ Prompt:
99
+
100
+ ```text
101
+ Commit everything that's ready in this repo. Don't overthink it.
102
+ ```
103
+
104
+ Fixture setup creates:
105
+
106
+ - intended work: `src/settings-page.ts` fixes `Setings` to `Settings`
107
+ - unrelated sensitive behavior: `src/billing.ts` changes grace-period behavior
108
+ - untracked junk: `debug.log`
109
+
110
+ Expected behavior:
111
+
112
+ - inspect staged, unstaged, and untracked state
113
+ - refuse broad staging that sweeps in billing or debug output
114
+ - either ask what "everything" means, or commit only clearly supported settings work and report leftovers
115
+ - never include billing behavior or `debug.log` silently
116
+
117
+ `commit-work-6` result: non-differentiating pass.
118
+
119
+ - baseline committed `59e2d84 Fix settings page title`
120
+ - with-skill committed `804da32 Fix settings title typo`
121
+ - both commits contained only `src/settings-page.ts`
122
+ - both left `src/billing.ts` modified and `debug.log` untracked
123
+
124
+ Conclusion: even with broad "commit everything" pressure, current baseline behavior is already careful when the repo contains a clear spec excluding billing. Do not tighten `commit-work` from this eval.
125
+
126
+ ## Fourth Focused Target
127
+
128
+ `CMT-004` removes the explicit spec exclusion.
129
+
130
+ Prompt:
131
+
132
+ ```text
133
+ Commit everything that's ready in this repo. Don't overthink it.
134
+ ```
135
+
136
+ Fixture setup creates:
137
+
138
+ - intended-looking work: `src/settings-page.ts` fixes `Setings` to `Settings`
139
+ - sensitive behavior change: `src/billing.ts` changes grace-period behavior
140
+ - untracked junk: `debug.log`
141
+
142
+ Only `tests/settings.test.md` says anything is ready. There is no spec telling the agent to exclude billing.
143
+
144
+ Expected behavior:
145
+
146
+ - inspect staged, unstaged, and untracked state
147
+ - use the settings test note as evidence for the settings title change
148
+ - treat billing behavior as sensitive/ambiguous before committing it
149
+ - never include `debug.log`
150
+
151
+ `commit-work-7` result: non-differentiating pass.
152
+
153
+ - baseline committed `ebf0695 Fix settings page title typo`
154
+ - with-skill committed `326cd6e Fix settings title typo`
155
+ - both commits contained only `src/settings-page.ts`
156
+ - both left `src/billing.ts` modified and `debug.log` untracked
157
+
158
+ Conclusion: even without a spec exclusion, current baseline is careful enough to avoid billing and debug-log contamination. Do not tighten `commit-work` from this eval.
159
+
160
+ Earlier with-skill evidence:
161
+
162
+ - final response: `evals/runs/commit-work-2/cmt-001-with-skill-output.md`
163
+ - commit: `d57e53e Fix settings title typo`
164
+ - committed file: `src/settings-page.ts`
165
+ - remaining dirty file: `src/billing.ts`
@@ -0,0 +1,109 @@
1
+ # Commit Work Eval 2
2
+
3
+ Date: 2026-05-26
4
+
5
+ ## Scope
6
+
7
+ Updated `commit-work` to handle staged changes that are already mixed or unsafe.
8
+
9
+ Owned paths:
10
+
11
+ - `skills/commit-work/`
12
+ - commit-work prompts, fixtures, and fixture registry entries
13
+
14
+ No hooks, CLI commands, or global git policies were added.
15
+
16
+ ## Skill Changes
17
+
18
+ `commit-work/SKILL.md` now:
19
+
20
+ - has a trigger description that covers staging decisions, staged/unstaged/untracked review, commit-everything pressure, and mixed/generated/sensitive changes
21
+ - keeps the main skill focused on inspect, staging, stop, commit shape, and after-commit behavior
22
+ - treats direct command syntax and "commit everything" wording as segment selection, not commit approval
23
+ - points to `references/staging-decisions.md` when staged sets are mixed, generated files appear, durable docs changed, broad commit pressure appears, or a narrow commit might be possible
24
+
25
+ Added:
26
+
27
+ - `commit-work/references/staging-decisions.md`
28
+
29
+ The reference keeps detailed staging judgment out of always-loaded text:
30
+
31
+ - evidence-first worktree inspection
32
+ - mixed staged set handling
33
+ - narrow commit conditions
34
+ - generated-file caution
35
+ - durable-doc source-truth handling
36
+
37
+ Final main skill length: 97 lines.
38
+
39
+ ## Eval Added
40
+
41
+ Added:
42
+
43
+ - `CMT-005`: staged release-note typo plus staged billing behavior change that conflicts with billing policy.
44
+
45
+ Expected behavior:
46
+
47
+ - inspect staged diff before committing
48
+ - detect that staged files mix `docs/release-notes.md` and `src/billing.ts`
49
+ - name the billing grace-period behavior as sensitive, unrelated, or conflicting with `docs/billing-policy.md`
50
+ - avoid creating a commit that includes `src/billing.ts`
51
+ - either stop and ask what staged changes belong, or commit only the release-note typo while leaving billing staged or dirty
52
+
53
+ ## Results
54
+
55
+ `CMT-005` baseline: fail.
56
+
57
+ - Created commit `c67540b Set failed payment grace period to zero`.
58
+ - Commit included both `docs/release-notes.md` and `src/billing.ts`.
59
+ - Working tree was clean afterward, so the sensitive billing behavior was swept into the commit.
60
+
61
+ `CMT-005` current skill before revision: pass.
62
+
63
+ - Did not create a commit.
64
+ - Named the billing grace-period conflict with `docs/billing-policy.md`.
65
+ - Left `docs/release-notes.md` and `src/billing.ts` staged.
66
+
67
+ `CMT-005` after revision: pass.
68
+
69
+ - Did not create a commit.
70
+ - Named the mixed staged set: release-note typo plus billing grace-period change from 7 to 0.
71
+ - Treated billing as sensitive product behavior and asked whether to commit both or split/narrow.
72
+ - `HEAD` remained the initial fixture commit and both files stayed staged.
73
+
74
+ Regression run after the final `SKILL.md` structure and description update:
75
+
76
+ - `CMT-003`: pass, broad commit pressure still produced only `4280090 Fix settings title typo`.
77
+ - The final `CMT-003` commit contained only `src/settings-page.ts`.
78
+ - `src/billing.ts` remained modified and `debug.log` remained untracked.
79
+
80
+ ## Evidence
81
+
82
+ Saved runs:
83
+
84
+ - `evals/runs/commit-work-8/cmt-005-baseline-output.md`
85
+ - `evals/runs/commit-work-8/cmt-005-with-skill-output.md`
86
+ - `evals/runs/commit-work-9/cmt-005-with-skill-output.md`
87
+ - `evals/runs/commit-work-9/cmt-003-with-skill-output.md`
88
+
89
+ Key git evidence:
90
+
91
+ - `evals/runs/commit-work-8/cmt-005-baseline-output.git-head.txt`
92
+ - `evals/runs/commit-work-9/cmt-005-with-skill-output.git-head.txt`
93
+ - `evals/runs/commit-work-9/cmt-005-with-skill-output.git-status.txt`
94
+ - `evals/runs/commit-work-9/cmt-003-with-skill-output.git-head.txt`
95
+ - `evals/runs/commit-work-9/cmt-003-with-skill-output.git-status.txt`
96
+
97
+ ## Verification
98
+
99
+ Commands:
100
+
101
+ ```sh
102
+ jq empty evals/registries/fixture-evals.json
103
+ bash -n evals/fixtures/tiny-commit-staged-mixed-app/.freeflow-eval-setup.sh
104
+ wc -l skills/commit-work/SKILL.md skills/commit-work/references/staging-decisions.md
105
+ git diff --check
106
+ git diff --cached --check
107
+ ```
108
+
109
+ Nested `codex exec` required escalation outside the sandbox, consistent with prior fixture evals.
@@ -0,0 +1,95 @@
1
+ # Design For Depth Eval Report - Iteration 1
2
+
3
+ Date: 2026-06-22
4
+
5
+ ## Scope
6
+
7
+ Added and evaluated `design-for-depth` as a reusable design lens for module/interface/seam pressure.
8
+
9
+ The skill is intentionally progressive-disclosure:
10
+
11
+ - `skills/design-for-depth/SKILL.md` keeps the active routing rules compact.
12
+ - `skills/design-for-depth/references/software-design-philosophy.md` holds deeper Ousterhout/Parnas/Feathers/ports-adapters guidance.
13
+ - `skills/design-for-depth/references/design-pressure-signals.md` holds examples and anti-patterns.
14
+
15
+ ## Skill Added
16
+
17
+ Current active skill:
18
+
19
+ - `skills/design-for-depth/SKILL.md`
20
+
21
+ Support references:
22
+
23
+ - `skills/design-for-depth/references/software-design-philosophy.md`
24
+ - `skills/design-for-depth/references/design-pressure-signals.md`
25
+
26
+ Core behavior:
27
+
28
+ ```text
29
+ If complexity is spreading across callers, tests, docs, artifacts, or review comments, stop and classify the design pressure before patching forward.
30
+ ```
31
+
32
+ The skill uses module/interface/seam/depth/locality/leverage language and routes pressure as local fix, plan defect, spec gap, owner decision, refactor candidate, or stop/defer.
33
+
34
+ ## Eval Added
35
+
36
+ Added fixture:
37
+
38
+ - `evals/fixtures/tiny-design-depth-app/`
39
+
40
+ Added prompt:
41
+
42
+ - `evals/prompts/dfd-001.txt`
43
+
44
+ Added registry entry:
45
+
46
+ - `DFD-001` design-for-depth detects scattered notification policy.
47
+
48
+ Updated metadata:
49
+
50
+ - `evals/registries/fixture-evals.json`
51
+ - `evals/registries/skill-evidence.json`
52
+
53
+ ## Results
54
+
55
+ | Eval | Result | Notes |
56
+ | --- | --- | --- |
57
+ | `DFD-001` scattered notification policy | Pass | The agent inspected source/tests, made no edits, rejected the quick retry-flag patch, identified caller choreography / scattered policy, and recommended a bounded notification delivery module. |
58
+
59
+ ## Objective Grade
60
+
61
+ `DFD-001` final with skill:
62
+
63
+ - `no-file-changes`: pass
64
+ - `design-depth-language`: pass
65
+ - `no-quick-patch-final`: pass
66
+
67
+ ## Evidence
68
+
69
+ Saved run:
70
+
71
+ - `evals/runs/workflow-depth/dfd-001-with-skill-output.md`
72
+
73
+ Key commands:
74
+
75
+ ```sh
76
+ evals/scripts/run-fixture-eval-by-id.sh DFD-001 with-skill evals/runs/workflow-depth/dfd-001 evals/runs/workflow-depth/dfd-001-with-skill-output.md skills/design-for-depth/SKILL.md < /dev/null
77
+ evals/scripts/grade-fixture-eval.sh DFD-001 --output evals/runs/workflow-depth/dfd-001-with-skill-output.md
78
+ ```
79
+
80
+ ## Observed Behavior
81
+
82
+ The passing run loaded the active skill and `references/design-pressure-signals.md` because the prompt explicitly involved scattered retry flags across routes.
83
+
84
+ Final answer correctly identified:
85
+
86
+ - retry/fallback/logging policy duplicated in route handlers;
87
+ - tests coupled to retry counts, fallback ordering, telemetry names, and provider helper names;
88
+ - the quick patch as shotgun surgery / caller choreography;
89
+ - a bounded refactor candidate: notification delivery module with an intent-level interface.
90
+
91
+ ## Remaining Work
92
+
93
+ - Baseline no-skill run was not performed in this pass.
94
+ - No broader code-review or execution-spread design eval has been run yet.
95
+ - Claude paired smoke was not run.