@hassangameryt/freeflow 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (630) hide show
  1. package/.agents/plugins/marketplace.json +1 -1
  2. package/.claude-plugin/marketplace.json +2 -2
  3. package/{plugins/freeflow/.claude-plugin → .claude-plugin}/plugin.json +1 -1
  4. package/{plugins/freeflow/.codex-plugin → .codex-plugin}/plugin.json +1 -1
  5. package/README.md +25 -16
  6. package/assets/freeflow-plugin-icon.png +0 -0
  7. package/{plugins/freeflow/command-surface.json → command-surface.json} +10 -10
  8. package/evals/README.md +32 -0
  9. package/evals/fixtures/copy-conventions.md +31 -0
  10. package/evals/fixtures/output-router/README.md +12 -0
  11. package/evals/fixtures/output-router/failed-command-output.txt +9 -0
  12. package/evals/fixtures/output-router/large-router-manual.md +63 -0
  13. package/evals/fixtures/output-router/native-large-output.txt +40 -0
  14. package/evals/fixtures/output-router/noisy-test-output.txt +30 -0
  15. package/evals/fixtures/output-router/verification-output.txt +8 -0
  16. package/evals/fixtures/tiny-artifact-review-app/README.md +5 -0
  17. package/evals/fixtures/tiny-artifact-review-app/docs/billing-policy.md +10 -0
  18. package/evals/fixtures/tiny-artifact-review-app/docs/specs/billing-webhook-api-draft.md +21 -0
  19. package/evals/fixtures/tiny-artifact-review-app/docs/specs/failed-payment-grace-period.md +24 -0
  20. package/evals/fixtures/tiny-artifact-review-app/docs/specs/immediate-downgrade.md +17 -0
  21. package/evals/fixtures/tiny-artifact-review-app/tests/billing.test.md +8 -0
  22. package/evals/fixtures/tiny-commit-ambiguous-app/.freeflow-eval-setup.sh +15 -0
  23. package/evals/fixtures/tiny-commit-ambiguous-app/README.md +3 -0
  24. package/evals/fixtures/tiny-commit-ambiguous-app/src/billing.ts +5 -0
  25. package/evals/fixtures/tiny-commit-ambiguous-app/src/settings-page.ts +1 -0
  26. package/evals/fixtures/tiny-commit-ambiguous-app/tests/settings.test.md +5 -0
  27. package/evals/fixtures/tiny-commit-app/.freeflow-eval-setup.sh +14 -0
  28. package/evals/fixtures/tiny-commit-app/README.md +3 -0
  29. package/evals/fixtures/tiny-commit-app/docs/specs/settings-title.md +5 -0
  30. package/evals/fixtures/tiny-commit-app/src/billing.ts +5 -0
  31. package/evals/fixtures/tiny-commit-app/src/settings-page.ts +1 -0
  32. package/evals/fixtures/tiny-commit-app/tests/settings.test.md +5 -0
  33. package/evals/fixtures/tiny-commit-everything-app/.freeflow-eval-setup.sh +15 -0
  34. package/evals/fixtures/tiny-commit-everything-app/README.md +3 -0
  35. package/evals/fixtures/tiny-commit-everything-app/docs/specs/settings-title.md +5 -0
  36. package/evals/fixtures/tiny-commit-everything-app/src/billing.ts +5 -0
  37. package/evals/fixtures/tiny-commit-everything-app/src/settings-page.ts +1 -0
  38. package/evals/fixtures/tiny-commit-everything-app/tests/settings.test.md +5 -0
  39. package/evals/fixtures/tiny-commit-staged-app/.freeflow-eval-setup.sh +15 -0
  40. package/evals/fixtures/tiny-commit-staged-app/README.md +3 -0
  41. package/evals/fixtures/tiny-commit-staged-app/docs/specs/settings-title.md +5 -0
  42. package/evals/fixtures/tiny-commit-staged-app/src/billing.ts +5 -0
  43. package/evals/fixtures/tiny-commit-staged-app/src/settings-page.ts +1 -0
  44. package/evals/fixtures/tiny-commit-staged-app/tests/settings.test.md +5 -0
  45. package/evals/fixtures/tiny-commit-staged-mixed-app/.freeflow-eval-setup.sh +15 -0
  46. package/evals/fixtures/tiny-commit-staged-mixed-app/README.md +3 -0
  47. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/billing-policy.md +3 -0
  48. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/release-notes.md +3 -0
  49. package/evals/fixtures/tiny-commit-staged-mixed-app/src/billing.ts +5 -0
  50. package/evals/fixtures/tiny-dashboard-stale-app/README.md +11 -0
  51. package/evals/fixtures/tiny-dashboard-stale-app/src/api-client.ts +13 -0
  52. package/evals/fixtures/tiny-dashboard-stale-app/src/dashboard-data.ts +19 -0
  53. package/evals/fixtures/tiny-dashboard-stale-app/tests/dashboard-data.test.md +9 -0
  54. package/evals/fixtures/tiny-decision-destination-app/README.md +6 -0
  55. package/evals/fixtures/tiny-decision-destination-app/src/risk.ts +8 -0
  56. package/evals/fixtures/tiny-decision-destination-app/tests/risk.test.md +4 -0
  57. package/evals/fixtures/tiny-design-depth-app/README.md +15 -0
  58. package/evals/fixtures/tiny-design-depth-app/src/routes.ts +25 -0
  59. package/evals/fixtures/tiny-design-depth-app/src/sendEmail.ts +3 -0
  60. package/evals/fixtures/tiny-design-depth-app/src/sendSms.ts +3 -0
  61. package/evals/fixtures/tiny-design-depth-app/src/telemetry.ts +3 -0
  62. package/evals/fixtures/tiny-design-depth-app/tests/routes.test.md +12 -0
  63. package/evals/fixtures/tiny-eval-method-app/README.md +8 -0
  64. package/evals/fixtures/tiny-eval-method-app/evals/reports/review-pr-eval-result.md +5 -0
  65. package/evals/fixtures/tiny-eval-method-app/evals/reports/triage-question-failure.md +17 -0
  66. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.diff +11 -0
  67. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.md +3 -0
  68. package/evals/fixtures/tiny-eval-method-app/skills/review-pr/SKILL.md +10 -0
  69. package/evals/fixtures/tiny-eval-method-app/skills/triage-question/SKILL.md +10 -0
  70. package/evals/fixtures/tiny-execute-plan-app/README.md +5 -0
  71. package/evals/fixtures/tiny-execute-plan-app/checks/settings-and-billing-check.sh +5 -0
  72. package/evals/fixtures/tiny-execute-plan-app/docs/billing-policy.md +10 -0
  73. package/evals/fixtures/tiny-execute-plan-app/docs/plans/immediate-downgrade-plan.md +11 -0
  74. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-plan.md +12 -0
  75. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-with-bad-check.md +15 -0
  76. package/evals/fixtures/tiny-execute-plan-app/docs/plans/unverified-billing-plan.md +9 -0
  77. package/evals/fixtures/tiny-execute-plan-app/docs/specs/failed-payment-grace-period.md +9 -0
  78. package/evals/fixtures/tiny-execute-plan-app/docs/specs/settings-title.md +9 -0
  79. package/evals/fixtures/tiny-execute-plan-app/src/billing.ts +7 -0
  80. package/evals/fixtures/tiny-execute-plan-app/src/settings-page.ts +1 -0
  81. package/evals/fixtures/tiny-execute-plan-app/tests/billing.test.md +7 -0
  82. package/evals/fixtures/tiny-execute-plan-app/tests/settings.test.md +5 -0
  83. package/evals/fixtures/tiny-execute-review-loop-app/README.md +5 -0
  84. package/evals/fixtures/tiny-execute-review-loop-app/docs/billing-policy.md +9 -0
  85. package/evals/fixtures/tiny-execute-review-loop-app/docs/plans/settings-title-plan.md +12 -0
  86. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/previous-agent-summary.md +15 -0
  87. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/review-pass-1-non-pass.md +11 -0
  88. package/evals/fixtures/tiny-execute-review-loop-app/docs/specs/settings-title.md +9 -0
  89. package/evals/fixtures/tiny-execute-review-loop-app/src/billing.ts +7 -0
  90. package/evals/fixtures/tiny-execute-review-loop-app/src/settings-page.ts +1 -0
  91. package/evals/fixtures/tiny-execute-review-loop-app/tests/billing.test.md +7 -0
  92. package/evals/fixtures/tiny-execute-review-loop-app/tests/settings.test.md +5 -0
  93. package/evals/fixtures/tiny-flaky-session-app/README.md +10 -0
  94. package/evals/fixtures/tiny-flaky-session-app/src/session-timeout.ts +13 -0
  95. package/evals/fixtures/tiny-flaky-session-app/tests/session-timeout.test.md +8 -0
  96. package/evals/fixtures/tiny-handoff-app/README.md +5 -0
  97. package/evals/fixtures/tiny-handoff-app/docs/billing-policy.md +10 -0
  98. package/evals/fixtures/tiny-handoff-app/docs/handoffs/old-agent-handoff.md +8 -0
  99. package/evals/fixtures/tiny-handoff-app/docs/plans/billing-retry-plan.md +11 -0
  100. package/evals/fixtures/tiny-handoff-app/src/billing.ts +7 -0
  101. package/evals/fixtures/tiny-handoff-report-app/README.md +3 -0
  102. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.diff +1 -0
  103. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.md +1 -0
  104. package/evals/fixtures/tiny-handoff-report-app/evals/scores/hof-003-score.txt +4 -0
  105. package/evals/fixtures/tiny-plan-app/README.md +5 -0
  106. package/evals/fixtures/tiny-plan-app/docs/billing-policy.md +10 -0
  107. package/evals/fixtures/tiny-plan-app/docs/specs/failed-payment-grace-period.md +25 -0
  108. package/evals/fixtures/tiny-plan-app/src/billing.ts +17 -0
  109. package/evals/fixtures/tiny-plan-app/src/settings-page.ts +1 -0
  110. package/evals/fixtures/tiny-plan-app/tests/billing.test.md +8 -0
  111. package/evals/fixtures/tiny-plan-app/tests/settings.test.md +5 -0
  112. package/evals/fixtures/tiny-plan-strict-api-app/README.md +3 -0
  113. package/evals/fixtures/tiny-plan-strict-api-app/docs/billing-policy.md +5 -0
  114. package/evals/fixtures/tiny-plan-strict-api-app/docs/specs/billing-webhook-api.md +25 -0
  115. package/evals/fixtures/tiny-plan-strict-api-app/src/billing.ts +5 -0
  116. package/evals/fixtures/tiny-plan-strict-api-app/src/webhooks.ts +3 -0
  117. package/evals/fixtures/tiny-plan-strict-api-app/tests/billing-webhook.test.md +8 -0
  118. package/evals/fixtures/tiny-post-setup-invalid-config-app/.freeflow/config.json +3 -0
  119. package/evals/fixtures/tiny-post-setup-invalid-config-app/AGENTS.md +15 -0
  120. package/evals/fixtures/tiny-post-setup-invalid-config-app/README.md +16 -0
  121. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/auth-review.md +15 -0
  122. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/billing-policy.md +10 -0
  123. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/copy-conventions.md +15 -0
  124. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/auth-middleware.ts +12 -0
  125. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/billing.ts +20 -0
  126. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/login-copy.ts +1 -0
  127. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/settings-page.ts +1 -0
  128. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/auth-middleware.test.md +8 -0
  129. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/billing.test.md +8 -0
  130. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/copy.test.md +7 -0
  131. package/evals/fixtures/tiny-post-setup-missing-config-app/AGENTS.md +15 -0
  132. package/evals/fixtures/tiny-post-setup-missing-config-app/README.md +16 -0
  133. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/auth-review.md +15 -0
  134. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/billing-policy.md +10 -0
  135. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/copy-conventions.md +15 -0
  136. package/evals/fixtures/tiny-post-setup-missing-config-app/src/auth-middleware.ts +12 -0
  137. package/evals/fixtures/tiny-post-setup-missing-config-app/src/billing.ts +20 -0
  138. package/evals/fixtures/tiny-post-setup-missing-config-app/src/login-copy.ts +1 -0
  139. package/evals/fixtures/tiny-post-setup-missing-config-app/src/settings-page.ts +1 -0
  140. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/auth-middleware.test.md +8 -0
  141. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/billing.test.md +8 -0
  142. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/copy.test.md +7 -0
  143. package/evals/fixtures/tiny-post-setup-source-conflict-app/.freeflow/config.json +3 -0
  144. package/evals/fixtures/tiny-post-setup-source-conflict-app/AGENTS.md +15 -0
  145. package/evals/fixtures/tiny-post-setup-source-conflict-app/README.md +16 -0
  146. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/auth-review.md +15 -0
  147. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/billing-policy.md +10 -0
  148. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/copy-conventions.md +15 -0
  149. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/auth-middleware.ts +12 -0
  150. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/billing.ts +20 -0
  151. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/login-copy.ts +1 -0
  152. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/settings-page.ts +1 -0
  153. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/auth-middleware.test.md +8 -0
  154. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/billing.test.md +8 -0
  155. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/copy.test.md +7 -0
  156. package/evals/fixtures/tiny-report-performance-app/README.md +11 -0
  157. package/evals/fixtures/tiny-report-performance-app/docs/performance.md +10 -0
  158. package/evals/fixtures/tiny-report-performance-app/src/report-summary.ts +29 -0
  159. package/evals/fixtures/tiny-report-performance-app/tests/report-performance.test.md +8 -0
  160. package/evals/fixtures/tiny-review-request-app/README.md +3 -0
  161. package/evals/fixtures/tiny-review-request-app/docs/billing-policy.md +5 -0
  162. package/evals/fixtures/tiny-review-request-app/docs/plans/billing-webhook-api.md +19 -0
  163. package/evals/fixtures/tiny-review-request-app/docs/review/previous-agent-summary.md +20 -0
  164. package/evals/fixtures/tiny-review-request-app/docs/specs/billing-webhook-api.md +25 -0
  165. package/evals/fixtures/tiny-review-request-app/src/billing.ts +11 -0
  166. package/evals/fixtures/tiny-review-request-app/src/webhooks.ts +19 -0
  167. package/evals/fixtures/tiny-review-request-app/tests/billing-webhook.test.md +8 -0
  168. package/evals/fixtures/tiny-review-work-app/README.md +5 -0
  169. package/evals/fixtures/tiny-review-work-app/docs/billing-policy.md +9 -0
  170. package/evals/fixtures/tiny-review-work-app/docs/plans/settings-title-plan.md +12 -0
  171. package/evals/fixtures/tiny-review-work-app/docs/review/previous-agent-summary.md +15 -0
  172. package/evals/fixtures/tiny-review-work-app/docs/specs/settings-title.md +9 -0
  173. package/evals/fixtures/tiny-review-work-app/src/billing.ts +7 -0
  174. package/evals/fixtures/tiny-review-work-app/src/settings-page.ts +1 -0
  175. package/evals/fixtures/tiny-review-work-app/tests/billing.test.md +7 -0
  176. package/evals/fixtures/tiny-review-work-app/tests/settings.test.md +5 -0
  177. package/evals/fixtures/tiny-saas-app/README.md +16 -0
  178. package/evals/fixtures/tiny-saas-app/docs/auth-review.md +15 -0
  179. package/evals/fixtures/tiny-saas-app/docs/billing-policy.md +10 -0
  180. package/evals/fixtures/tiny-saas-app/docs/copy-conventions.md +15 -0
  181. package/evals/fixtures/tiny-saas-app/docs/specs/settings-title.md +13 -0
  182. package/evals/fixtures/tiny-saas-app/src/auth-middleware.ts +12 -0
  183. package/evals/fixtures/tiny-saas-app/src/billing.ts +20 -0
  184. package/evals/fixtures/tiny-saas-app/src/login-copy.ts +1 -0
  185. package/evals/fixtures/tiny-saas-app/src/settings-page.ts +1 -0
  186. package/evals/fixtures/tiny-saas-app/tests/auth-middleware.test.md +8 -0
  187. package/evals/fixtures/tiny-saas-app/tests/billing.test.md +8 -0
  188. package/evals/fixtures/tiny-saas-app/tests/copy.test.md +7 -0
  189. package/evals/fixtures/tiny-setup-agents-app/AGENTS.md +6 -0
  190. package/evals/fixtures/tiny-setup-both-app/AGENTS.md +5 -0
  191. package/evals/fixtures/tiny-setup-both-app/CLAUDE.md +5 -0
  192. package/evals/fixtures/tiny-setup-claude-app/CLAUDE.md +6 -0
  193. package/evals/fixtures/tiny-setup-codex-rules-app/.codex/rules/shell.rules +5 -0
  194. package/evals/fixtures/tiny-setup-codex-rules-app/AGENTS.md +6 -0
  195. package/evals/fixtures/tiny-setup-conflict-app/AGENTS.md +6 -0
  196. package/evals/fixtures/tiny-setup-existing-block-app/AGENTS.md +14 -0
  197. package/evals/fixtures/tiny-setup-four-rules-app/CLAUDE.md +6 -0
  198. package/evals/fixtures/tiny-skill-dev-app/README.md +10 -0
  199. package/evals/fixtures/tiny-skill-dev-app/evals/prompts/review-pr-001.txt +8 -0
  200. package/evals/fixtures/tiny-skill-dev-app/evals/reports/review-pr-failure.md +17 -0
  201. package/evals/fixtures/tiny-skill-dev-app/skills/review-pr/SKILL.md +13 -0
  202. package/evals/fixtures/tiny-team-app/README.md +5 -0
  203. package/evals/fixtures/tiny-team-app/docs/product.md +6 -0
  204. package/evals/fixtures/tiny-team-app/src/teams.ts +9 -0
  205. package/evals/fixtures/tiny-team-app/src/users.ts +5 -0
  206. package/evals/fixtures/tiny-team-app/tests/teams.test.md +7 -0
  207. package/evals/fixtures/tiny-verify-app/README.md +5 -0
  208. package/evals/fixtures/tiny-verify-app/src/settings-nav.ts +1 -0
  209. package/evals/fixtures/tiny-verify-app/src/settings-page.ts +1 -0
  210. package/evals/fixtures/tiny-verify-app/tests/settings.test.md +7 -0
  211. package/evals/prompts/aon-001.txt +2 -0
  212. package/evals/prompts/byp-001.txt +2 -0
  213. package/evals/prompts/byp-002.txt +2 -0
  214. package/evals/prompts/cap-001.txt +3 -0
  215. package/evals/prompts/cap-002.txt +3 -0
  216. package/evals/prompts/cmd-001.txt +3 -0
  217. package/evals/prompts/cmd-002.txt +3 -0
  218. package/evals/prompts/cmd-003.txt +6 -0
  219. package/evals/prompts/cmd-004.txt +5 -0
  220. package/evals/prompts/cmd-005.txt +5 -0
  221. package/evals/prompts/cmd-006.txt +5 -0
  222. package/evals/prompts/cmd-007.txt +5 -0
  223. package/evals/prompts/cmd-008.txt +5 -0
  224. package/evals/prompts/cmd-009.txt +8 -0
  225. package/evals/prompts/cmd-010.txt +5 -0
  226. package/evals/prompts/cmd-011.txt +5 -0
  227. package/evals/prompts/cmd-012.txt +5 -0
  228. package/evals/prompts/cmd-013.txt +5 -0
  229. package/evals/prompts/cmd-014.txt +7 -0
  230. package/evals/prompts/cmd-015.txt +7 -0
  231. package/evals/prompts/cmd-016.txt +5 -0
  232. package/evals/prompts/cmt-001.txt +1 -0
  233. package/evals/prompts/cmt-002.txt +1 -0
  234. package/evals/prompts/cmt-003.txt +1 -0
  235. package/evals/prompts/cmt-004.txt +1 -0
  236. package/evals/prompts/cmt-005.txt +1 -0
  237. package/evals/prompts/dfd-001.txt +3 -0
  238. package/evals/prompts/dia-001.txt +3 -0
  239. package/evals/prompts/dia-002.txt +3 -0
  240. package/evals/prompts/dia-003.txt +3 -0
  241. package/evals/prompts/dis-001.txt +5 -0
  242. package/evals/prompts/dis-002.txt +5 -0
  243. package/evals/prompts/esk-001.txt +5 -0
  244. package/evals/prompts/esk-002.txt +5 -0
  245. package/evals/prompts/esk-003.txt +5 -0
  246. package/evals/prompts/grc-001.txt +3 -0
  247. package/evals/prompts/hof-001.txt +3 -0
  248. package/evals/prompts/hof-002.txt +1 -0
  249. package/evals/prompts/hof-003.txt +1 -0
  250. package/evals/prompts/hof-004.txt +1 -0
  251. package/evals/prompts/hof-005.txt +1 -0
  252. package/evals/prompts/ivg-001.txt +1 -0
  253. package/evals/prompts/ivg-002.txt +11 -0
  254. package/evals/prompts/ivg-003.txt +1 -0
  255. package/evals/prompts/ivg-004.txt +1 -0
  256. package/evals/prompts/mode-001.txt +3 -0
  257. package/evals/prompts/mode-002.txt +1 -0
  258. package/evals/prompts/mode-003.txt +3 -0
  259. package/evals/prompts/mode-004.txt +3 -0
  260. package/evals/prompts/mode-005.txt +3 -0
  261. package/evals/prompts/mode-006.txt +3 -0
  262. package/evals/prompts/mode-007.txt +3 -0
  263. package/evals/prompts/otr-001.txt +3 -0
  264. package/evals/prompts/otr-002.txt +3 -0
  265. package/evals/prompts/rar-001.txt +5 -0
  266. package/evals/prompts/rar-002.txt +3 -0
  267. package/evals/prompts/rar-003.txt +6 -0
  268. package/evals/prompts/rar-004.txt +15 -0
  269. package/evals/prompts/rar-005.txt +10 -0
  270. package/evals/prompts/rbr-001.txt +3 -0
  271. package/evals/prompts/rbr-002.txt +3 -0
  272. package/evals/prompts/rbr-003.txt +7 -0
  273. package/evals/prompts/rev-001.txt +3 -0
  274. package/evals/prompts/rev-002.txt +5 -0
  275. package/evals/prompts/rev-003.txt +6 -0
  276. package/evals/prompts/rev-004.txt +1 -0
  277. package/evals/prompts/rev-005.txt +15 -0
  278. package/evals/prompts/rev-006.txt +5 -0
  279. package/evals/prompts/stp-001.txt +1 -0
  280. package/evals/prompts/stp-002.txt +1 -0
  281. package/evals/prompts/stp-003.txt +1 -0
  282. package/evals/prompts/stp-004.txt +1 -0
  283. package/evals/prompts/stp-005.txt +1 -0
  284. package/evals/prompts/stp-006.txt +1 -0
  285. package/evals/prompts/stp-007.txt +1 -0
  286. package/evals/prompts/stp-008.txt +1 -0
  287. package/evals/prompts/stp-009.txt +1 -0
  288. package/evals/prompts/stp-010.txt +1 -0
  289. package/evals/prompts/stp-011.txt +1 -0
  290. package/evals/prompts/vfy-003.txt +1 -0
  291. package/evals/prompts/wpl-001.txt +3 -0
  292. package/evals/prompts/wpl-002.txt +5 -0
  293. package/evals/prompts/wpl-003.txt +3 -0
  294. package/evals/prompts/wpl-004.txt +5 -0
  295. package/evals/prompts/wpl-005.txt +1 -0
  296. package/evals/prompts/wrc-001.txt +10 -0
  297. package/evals/prompts/wrc-002.txt +3 -0
  298. package/evals/prompts/wsk-001.txt +6 -0
  299. package/evals/prompts/wsp-001.txt +5 -0
  300. package/evals/prompts/wsp-002.txt +15 -0
  301. package/evals/prompts/wsp-003.txt +3 -0
  302. package/evals/prompts/wsp-004.txt +15 -0
  303. package/evals/prompts/wsp-005.txt +5 -0
  304. package/evals/prompts/wsp-006.txt +12 -0
  305. package/evals/prompts/wsp-007.txt +9 -0
  306. package/evals/prompts/wsp-008.txt +13 -0
  307. package/evals/prompts/xpl-001.txt +1 -0
  308. package/evals/prompts/xpl-002.txt +3 -0
  309. package/evals/prompts/xpl-003.txt +3 -0
  310. package/evals/prompts/xpl-004.txt +3 -0
  311. package/evals/prompts/xpl-005.txt +5 -0
  312. package/evals/prompts/xpl-006.txt +3 -0
  313. package/evals/registries/adversarial-evals.json +39 -0
  314. package/evals/registries/fixture-evals.json +2513 -0
  315. package/evals/registries/handoff-evals.json +74 -0
  316. package/evals/registries/legacy-evals.json +147 -0
  317. package/evals/registries/skill-evidence.json +193 -0
  318. package/evals/registries/verify-work-evals.json +46 -0
  319. package/evals/reports/acceptance/v0.1-acceptance-report.md +130 -0
  320. package/evals/reports/by-command-surface/command-surface-1-report.md +45 -0
  321. package/evals/reports/by-command-surface/command-surface-10-report.md +76 -0
  322. package/evals/reports/by-command-surface/command-surface-11-report.md +77 -0
  323. package/evals/reports/by-command-surface/command-surface-12-report.md +91 -0
  324. package/evals/reports/by-command-surface/command-surface-13-report.md +64 -0
  325. package/evals/reports/by-command-surface/command-surface-14-report.md +66 -0
  326. package/evals/reports/by-command-surface/command-surface-15-report.md +124 -0
  327. package/evals/reports/by-command-surface/command-surface-16-report.md +39 -0
  328. package/evals/reports/by-command-surface/command-surface-2-report.md +80 -0
  329. package/evals/reports/by-command-surface/command-surface-3-report.md +86 -0
  330. package/evals/reports/by-command-surface/command-surface-4-report.md +76 -0
  331. package/evals/reports/by-command-surface/command-surface-5-report.md +80 -0
  332. package/evals/reports/by-command-surface/command-surface-6-report.md +74 -0
  333. package/evals/reports/by-command-surface/command-surface-7-report.md +84 -0
  334. package/evals/reports/by-command-surface/command-surface-8-report.md +79 -0
  335. package/evals/reports/by-command-surface/command-surface-9-report.md +79 -0
  336. package/evals/reports/by-command-surface/command-surface-matrix.md +91 -0
  337. package/evals/reports/by-skill/bypass-1-report.md +124 -0
  338. package/evals/reports/by-skill/capture-decisions-1-report.md +99 -0
  339. package/evals/reports/by-skill/capture-decisions-2-report.md +102 -0
  340. package/evals/reports/by-skill/commit-work-1-report.md +165 -0
  341. package/evals/reports/by-skill/commit-work-2-report.md +109 -0
  342. package/evals/reports/by-skill/design-for-depth-1-report.md +95 -0
  343. package/evals/reports/by-skill/diagnose-failure-1-report.md +91 -0
  344. package/evals/reports/by-skill/diagnose-failure-2-report.md +94 -0
  345. package/evals/reports/by-skill/discover-1-report.md +118 -0
  346. package/evals/reports/by-skill/evaluate-skill-2-report.md +71 -0
  347. package/evals/reports/by-skill/execute-plan-1-report.md +103 -0
  348. package/evals/reports/by-skill/execute-plan-2-report.md +112 -0
  349. package/evals/reports/by-skill/execute-plan-3-report.md +87 -0
  350. package/evals/reports/by-skill/grill-context-1-report.md +103 -0
  351. package/evals/reports/by-skill/handoff-1-report.md +123 -0
  352. package/evals/reports/by-skill/handoff-2-report.md +105 -0
  353. package/evals/reports/by-skill/handoff-3-report.md +100 -0
  354. package/evals/reports/by-skill/handoff-7-report.md +77 -0
  355. package/evals/reports/by-skill/interview-gate-1-report.md +63 -0
  356. package/evals/reports/by-skill/interview-gate-2-report.md +105 -0
  357. package/evals/reports/by-skill/mode-contract-1-report.md +74 -0
  358. package/evals/reports/by-skill/mode-contract-2-report.md +87 -0
  359. package/evals/reports/by-skill/mode-contract-3-report.md +65 -0
  360. package/evals/reports/by-skill/mode-contract-4-report.md +71 -0
  361. package/evals/reports/by-skill/mode-contract-5-report.md +70 -0
  362. package/evals/reports/by-skill/output-router-1-report.md +76 -0
  363. package/evals/reports/by-skill/output-router-2-report.md +65 -0
  364. package/evals/reports/by-skill/research-1-report.md +144 -0
  365. package/evals/reports/by-skill/research-brief-1-report.md +94 -0
  366. package/evals/reports/by-skill/research-brief-2-report.md +75 -0
  367. package/evals/reports/by-skill/review-artifact-1-report.md +91 -0
  368. package/evals/reports/by-skill/review-artifact-2-report.md +79 -0
  369. package/evals/reports/by-skill/review-artifact-3-report.md +120 -0
  370. package/evals/reports/by-skill/review-artifact-4-report.md +89 -0
  371. package/evals/reports/by-skill/review-artifact-5-report.md +66 -0
  372. package/evals/reports/by-skill/review-work-1-report.md +85 -0
  373. package/evals/reports/by-skill/review-work-2-report.md +87 -0
  374. package/evals/reports/by-skill/review-work-3-report.md +104 -0
  375. package/evals/reports/by-skill/review-work-4-report.md +108 -0
  376. package/evals/reports/by-skill/review-work-5-report.md +73 -0
  377. package/evals/reports/by-skill/review-work-6-report.md +66 -0
  378. package/evals/reports/by-skill/setup-freeflow-1-report.md +199 -0
  379. package/evals/reports/by-skill/setup-freeflow-2-report.md +105 -0
  380. package/evals/reports/by-skill/setup-freeflow-3-report.md +105 -0
  381. package/evals/reports/by-skill/setup-freeflow-4-report.md +88 -0
  382. package/evals/reports/by-skill/setup-freeflow-5-report.md +115 -0
  383. package/evals/reports/by-skill/skill-authoring-1-report.md +81 -0
  384. package/evals/reports/by-skill/verify-work-1-report.md +111 -0
  385. package/evals/reports/by-skill/verify-work-2-report.md +120 -0
  386. package/evals/reports/by-skill/workflow-1-report.md +65 -0
  387. package/evals/reports/by-skill/write-plan-1-report.md +123 -0
  388. package/evals/reports/by-skill/write-plan-2-report.md +98 -0
  389. package/evals/reports/by-skill/write-plan-3-report.md +110 -0
  390. package/evals/reports/by-skill/write-spec-1-report.md +88 -0
  391. package/evals/reports/by-skill/write-spec-2-report.md +89 -0
  392. package/evals/reports/by-skill/write-spec-3-report.md +90 -0
  393. package/evals/reports/by-skill/write-spec-4-report.md +167 -0
  394. package/evals/reports/harness/claude-harness-1-report.md +103 -0
  395. package/evals/reports/harness/eval-harness-1-report.md +91 -0
  396. package/evals/reports/iterations/iteration-1-report.md +71 -0
  397. package/evals/reports/iterations/iteration-2-report.md +78 -0
  398. package/evals/reports/iterations/iteration-3-report.md +118 -0
  399. package/evals/reports/iterations/iteration-4-report.md +131 -0
  400. package/evals/reports/iterations/iteration-5-report.md +149 -0
  401. package/evals/reports/iterations/iteration-6-report.md +136 -0
  402. package/evals/reports/runtime/always-on-runtime-1-report.md +107 -0
  403. package/evals/reports/runtime/context-mode-normalized-benchmark-1-report.md +52 -0
  404. package/evals/reports/runtime/context-mode-real-deep-baseline-1-report.md +410 -0
  405. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-report.md +203 -0
  406. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-review.md +51 -0
  407. package/evals/reports/runtime/context-saving-native-tools-baseline-1-report.md +99 -0
  408. package/evals/reports/runtime/eryx-python-compatibility-reprobe-1-report.md +166 -0
  409. package/evals/reports/runtime/eryx-python-proof-spike-1-report.md +96 -0
  410. package/evals/reports/runtime/eryx-python-proof-spike-2-report.md +151 -0
  411. package/evals/reports/runtime/eryx-python-script-derive-execution-1-report.md +233 -0
  412. package/evals/reports/runtime/jq-script-derive-execution-1-report.md +215 -0
  413. package/evals/reports/runtime/jq-wasm-proof-spike-1-report.md +67 -0
  414. package/evals/reports/runtime/output-router-benchmark-1-report.md +81 -0
  415. package/evals/reports/runtime/output-router-capture-derive-provider-eval-1-report.md +35 -0
  416. package/evals/reports/runtime/output-router-codex-large-index-sqlite-adhoc-report.md +150 -0
  417. package/evals/reports/runtime/output-router-codex-qa-benchmark-1-report.md +46 -0
  418. package/evals/reports/runtime/output-router-command-benchmark-1-report.md +62 -0
  419. package/evals/reports/runtime/output-router-index-benchmark-1-report.md +64 -0
  420. package/evals/reports/runtime/output-router-regression-1-report.md +81 -0
  421. package/evals/reports/runtime/output-router-transform-eval-1-report.md +28 -0
  422. package/evals/reports/runtime/pi-observed-routing-eval-1-report.md +44 -0
  423. package/evals/reports/runtime/quickjs-script-derive-execution-1-report.md +159 -0
  424. package/evals/reports/runtime/quickjs-wasi-proof-spike-1-report.md +55 -0
  425. package/evals/reports/runtime/render-format-row-protocol-1-report.md +58 -0
  426. package/evals/reports/runtime/script-sandbox-adapter-selection-review-1-report.md +84 -0
  427. package/evals/reports/runtime/script-sandbox-probe-resource-hardening-1-report.md +51 -0
  428. package/evals/reports/runtime/storage-policy-benchmark-1-report.md +176 -0
  429. package/evals/reports/runtime/vault-index-storage-spike-1-report.md +39 -0
  430. package/evals/reports/runtime/workflow-context-hook-1-report.md +82 -0
  431. package/evals/runbooks/adversarial-eval-runbook.md +28 -0
  432. package/evals/runbooks/fixture-eval-runbook.md +137 -0
  433. package/evals/runbooks/handoff-eval-runbook.md +22 -0
  434. package/evals/runbooks/verify-work-eval-runbook.md +27 -0
  435. package/evals/scripts/audit-command-surface.sh +160 -0
  436. package/evals/scripts/check-activation-contract.sh +121 -0
  437. package/evals/scripts/check-runtime-context-hook.sh +183 -0
  438. package/evals/scripts/grade-fixture-eval.sh +446 -0
  439. package/evals/scripts/run-claude-fixture-eval.sh +26 -0
  440. package/evals/scripts/run-codex-fixture-eval.sh +26 -0
  441. package/evals/scripts/run-eryx-python-proof-spike.js +559 -0
  442. package/evals/scripts/run-fixture-eval-by-id.sh +23 -0
  443. package/evals/scripts/run-fixture-eval.sh +464 -0
  444. package/evals/scripts/run-jq-wasm-proof-spike.js +299 -0
  445. package/evals/scripts/run-output-router-transform-eval.js +115 -0
  446. package/evals/scripts/run-pi-observed-routing-eval.js +478 -0
  447. package/evals/scripts/run-quickjs-wasi-proof-spike.js +339 -0
  448. package/evals/scripts/skill-evidence.sh +316 -0
  449. package/evals/scripts/validate-release-metadata.sh +486 -0
  450. package/evals/suites/v0.1-acceptance-suite.md +131 -0
  451. package/{plugins/freeflow/hooks → hooks}/freeflow-runtime-context.mjs +65 -14
  452. package/package.json +46 -13
  453. package/pi-extension/dist/host-producer-identification.js +186 -0
  454. package/pi-extension/dist/index.js +85 -0
  455. package/pi-extension/dist/mcp-config.js +30 -0
  456. package/pi-extension/dist/native-safety-net.js +209 -0
  457. package/pi-extension/dist/observed-tool-routing.js +114 -0
  458. package/pi-extension/dist/renderers.js +835 -0
  459. package/pi-extension/dist/router-tools.js +438 -0
  460. package/pi-extension/dist/runtime-context.js +288 -0
  461. package/pi-extension/dist/schemas.js +254 -0
  462. package/pi-extension/dist/status.js +449 -0
  463. package/pi-extension/dist/utils.js +467 -0
  464. package/pi-extension/src/host-producer-identification.ts +226 -0
  465. package/pi-extension/src/index.ts +121 -0
  466. package/pi-extension/src/mcp-config.ts +32 -0
  467. package/pi-extension/src/native-safety-net.ts +252 -0
  468. package/pi-extension/src/observed-tool-routing.ts +126 -0
  469. package/pi-extension/src/renderers.ts +917 -0
  470. package/pi-extension/src/router-tools.ts +478 -0
  471. package/pi-extension/src/runtime-context.ts +347 -0
  472. package/pi-extension/src/schemas.ts +259 -0
  473. package/pi-extension/src/status.ts +505 -0
  474. package/pi-extension/src/utils.ts +492 -0
  475. package/pi-extension/tsconfig.json +16 -0
  476. package/plugin-docs/README.md +40 -0
  477. package/plugin-docs/adr/0002-evals-before-hooks.md +17 -0
  478. package/{plugins/freeflow/docs → plugin-docs}/adr/0003-release-boundary.md +2 -2
  479. package/plugin-docs/adr/0004-discover-replaces-shallow-discovery-skills.md +26 -0
  480. package/plugin-docs/adr/README.md +10 -0
  481. package/plugin-docs/architecture.md +89 -0
  482. package/plugin-docs/output-router.md +396 -0
  483. package/plugin-docs/release-evidence.md +77 -0
  484. package/{plugins/freeflow/docs → plugin-docs}/skills.md +7 -7
  485. package/{plugins/freeflow/docs → plugin-docs}/workflow.md +4 -4
  486. package/router/dist/benchmarks/benchmark-harness.d.ts +32 -0
  487. package/router/dist/benchmarks/benchmark-harness.js +92 -0
  488. package/router/dist/benchmarks/benchmarks.d.ts +102 -0
  489. package/router/dist/benchmarks/benchmarks.js +880 -0
  490. package/router/dist/benchmarks/codex-qa-benchmarks.d.ts +91 -0
  491. package/router/dist/benchmarks/codex-qa-benchmarks.js +461 -0
  492. package/router/dist/benchmarks/command-benchmarks.d.ts +128 -0
  493. package/router/dist/benchmarks/command-benchmarks.js +653 -0
  494. package/router/dist/benchmarks/context-mode-normalized-benchmarks.d.ts +74 -0
  495. package/router/dist/benchmarks/context-mode-normalized-benchmarks.js +707 -0
  496. package/router/dist/benchmarks/context-mode-real-deep-benchmark.d.ts +124 -0
  497. package/router/dist/benchmarks/context-mode-real-deep-benchmark.js +1295 -0
  498. package/router/dist/benchmarks/context-saving-baseline.d.ts +54 -0
  499. package/router/dist/benchmarks/context-saving-baseline.js +439 -0
  500. package/router/dist/benchmarks/index-benchmarks.d.ts +98 -0
  501. package/router/dist/benchmarks/index-benchmarks.js +762 -0
  502. package/router/dist/benchmarks/storage-policy-benchmarks.d.ts +90 -0
  503. package/router/dist/benchmarks/storage-policy-benchmarks.js +538 -0
  504. package/router/dist/benchmarks/vault-index-benchmarks.d.ts +47 -0
  505. package/router/dist/benchmarks/vault-index-benchmarks.js +257 -0
  506. package/router/dist/config/config.d.ts +96 -0
  507. package/router/dist/config/config.js +453 -0
  508. package/router/dist/config/router-contract.d.ts +10 -0
  509. package/router/dist/config/router-contract.js +54 -0
  510. package/router/dist/config/schema.d.ts +18 -0
  511. package/router/dist/config/schema.js +600 -0
  512. package/router/dist/config/types.d.ts +521 -0
  513. package/router/dist/config/types.js +30 -0
  514. package/router/dist/evidence/bounded-evidence.d.ts +40 -0
  515. package/router/dist/evidence/bounded-evidence.js +310 -0
  516. package/router/dist/evidence/evidence-range-selector.d.ts +22 -0
  517. package/router/dist/evidence/evidence-range-selector.js +231 -0
  518. package/router/dist/evidence/evidence-search.d.ts +25 -0
  519. package/router/dist/evidence/evidence-search.js +466 -0
  520. package/router/dist/evidence/evidence.d.ts +38 -0
  521. package/router/dist/evidence/evidence.js +196 -0
  522. package/router/dist/evidence/failure-contracts.d.ts +27 -0
  523. package/router/dist/evidence/failure-contracts.js +160 -0
  524. package/router/dist/evidence/line-ranges.d.ts +22 -0
  525. package/router/dist/evidence/line-ranges.js +22 -0
  526. package/router/dist/experiments/local-index.d.ts +58 -0
  527. package/router/dist/experiments/local-index.js +562 -0
  528. package/router/dist/index.d.ts +30 -0
  529. package/router/dist/index.js +32 -0
  530. package/router/dist/processing/engine.d.ts +108 -0
  531. package/router/dist/processing/engine.js +435 -0
  532. package/router/dist/processing/reducers.d.ts +266 -0
  533. package/router/dist/processing/reducers.js +1495 -0
  534. package/router/dist/processing/renderers.d.ts +25 -0
  535. package/router/dist/processing/renderers.js +186 -0
  536. package/router/dist/processing/scripts.d.ts +78 -0
  537. package/router/dist/processing/scripts.js +333 -0
  538. package/router/dist/repo/repo-traversal.d.ts +18 -0
  539. package/router/dist/repo/repo-traversal.js +242 -0
  540. package/router/dist/routing/observed-reducers.d.ts +35 -0
  541. package/router/dist/routing/observed-reducers.js +647 -0
  542. package/router/dist/routing/observed-routing.d.ts +17 -0
  543. package/router/dist/routing/observed-routing.js +377 -0
  544. package/router/dist/routing/parsers.d.ts +16 -0
  545. package/router/dist/routing/parsers.js +473 -0
  546. package/router/dist/routing/run-filters.d.ts +56 -0
  547. package/router/dist/routing/run-filters.js +230 -0
  548. package/router/dist/routing/run-reducers.d.ts +30 -0
  549. package/router/dist/routing/run-reducers.js +110 -0
  550. package/router/dist/sandbox/adapter-roots.d.ts +38 -0
  551. package/router/dist/sandbox/adapter-roots.js +64 -0
  552. package/router/dist/sandbox/eryx-python-adapter.d.ts +9 -0
  553. package/router/dist/sandbox/eryx-python-adapter.js +797 -0
  554. package/router/dist/sandbox/jq-wasm-adapter.d.ts +9 -0
  555. package/router/dist/sandbox/jq-wasm-adapter.js +421 -0
  556. package/router/dist/sandbox/quickjs-wasi-adapter.d.ts +9 -0
  557. package/router/dist/sandbox/quickjs-wasi-adapter.js +381 -0
  558. package/router/dist/sandbox/script-sandbox.d.ts +113 -0
  559. package/router/dist/sandbox/script-sandbox.js +303 -0
  560. package/router/dist/setup/script-transform-adapters.d.ts +28 -0
  561. package/router/dist/setup/script-transform-adapters.js +259 -0
  562. package/router/dist/tools/batch.d.ts +22 -0
  563. package/router/dist/tools/batch.js +509 -0
  564. package/router/dist/tools/run.d.ts +50 -0
  565. package/router/dist/tools/run.js +1219 -0
  566. package/router/dist/tools/search.d.ts +44 -0
  567. package/router/dist/tools/search.js +1205 -0
  568. package/router/dist/transform/engine.d.ts +143 -0
  569. package/router/dist/transform/engine.js +1955 -0
  570. package/router/dist/vault/vault-index.d.ts +106 -0
  571. package/router/dist/vault/vault-index.js +411 -0
  572. package/router/dist/vault/vault.d.ts +103 -0
  573. package/router/dist/vault/vault.js +635 -0
  574. package/{plugins/freeflow/skills → skills}/commit-work/SKILL.md +17 -16
  575. package/{plugins/freeflow/skills → skills}/commit-work/references/staging-decisions.md +3 -3
  576. package/skills/design-for-depth/SKILL.md +90 -0
  577. package/skills/design-for-depth/references/design-pressure-signals.md +265 -0
  578. package/skills/design-for-depth/references/software-design-philosophy.md +219 -0
  579. package/{plugins/freeflow/skills → skills}/diagnose-failure/SKILL.md +6 -1
  580. package/skills/discover/ARTIFACT-DESTINATIONS.md +58 -0
  581. package/skills/discover/CHECKPOINTS.md +74 -0
  582. package/skills/discover/SKILL.md +173 -0
  583. package/skills/execute-plan/SKILL.md +133 -0
  584. package/skills/execute-plan/references/execution-map.md +122 -0
  585. package/{plugins/freeflow/skills → skills}/mode-contract/SKILL.md +10 -5
  586. package/skills/output-router/SKILL.md +202 -0
  587. package/skills/output-router/references/safety-policy.md +40 -0
  588. package/{plugins/freeflow/skills → skills}/review-artifact/SKILL.md +37 -1
  589. package/{plugins/freeflow/skills → skills}/review-artifact/references/reviewer-prompt.md +14 -2
  590. package/skills/review-work/SKILL.md +101 -0
  591. package/{plugins/freeflow/skills → skills}/review-work/references/reviewer-prompt.md +8 -0
  592. package/skills/setup-freeflow/SKILL.md +106 -0
  593. package/{plugins/freeflow/skills → skills}/setup-freeflow/references/activation-contract.md +14 -8
  594. package/{plugins/freeflow/skills → skills}/setup-freeflow/references/host-setup.md +5 -3
  595. package/skills/setup-freeflow/references/output-router-setup.md +102 -0
  596. package/{plugins/freeflow/skills → skills}/verify-work/SKILL.md +3 -1
  597. package/{plugins/freeflow/skills → skills}/workflow/SKILL.md +23 -13
  598. package/{plugins/freeflow/skills → skills}/workflow/references/workflow-map.md +21 -20
  599. package/{plugins/freeflow/skills → skills}/write-plan/SKILL.md +9 -1
  600. package/{plugins/freeflow/skills → skills}/write-skill/SKILL.md +1 -1
  601. package/{plugins/freeflow/skills → skills}/write-spec/SKILL.md +9 -6
  602. package/plugins/freeflow/docs/README.md +0 -39
  603. package/plugins/freeflow/docs/adr/0002-evals-before-hooks.md +0 -17
  604. package/plugins/freeflow/docs/adr/README.md +0 -7
  605. package/plugins/freeflow/docs/architecture.md +0 -86
  606. package/plugins/freeflow/docs/release-evidence.md +0 -49
  607. package/plugins/freeflow/pi-extension/index.js +0 -250
  608. package/plugins/freeflow/skills/capture-decisions/SKILL.md +0 -95
  609. package/plugins/freeflow/skills/capture-decisions/references/destination-guide.md +0 -65
  610. package/plugins/freeflow/skills/execute-plan/SKILL.md +0 -67
  611. package/plugins/freeflow/skills/grill-context/SKILL.md +0 -79
  612. package/plugins/freeflow/skills/grill-context/references/approach-framing.md +0 -52
  613. package/plugins/freeflow/skills/research-brief/SKILL.md +0 -69
  614. package/plugins/freeflow/skills/review-work/SKILL.md +0 -73
  615. package/plugins/freeflow/skills/setup-freeflow/SKILL.md +0 -84
  616. /package/{plugins/freeflow/assets → assets}/freeflow-plugin-icon-transparent.png +0 -0
  617. /package/{plugins/freeflow/hooks → hooks}/hooks.json +0 -0
  618. /package/{plugins/freeflow/docs → plugin-docs}/adr/0001-three-modes.md +0 -0
  619. /package/{plugins/freeflow/skills → skills}/bypass/SKILL.md +0 -0
  620. /package/{plugins/freeflow/skills → skills}/diagnose-failure/references/feedback-loop-catalog.md +0 -0
  621. /package/{plugins/freeflow/skills → skills}/diagnose-failure/references/flaky-and-performance.md +0 -0
  622. /package/{plugins/freeflow/skills → skills}/evaluate-skill/SKILL.md +0 -0
  623. /package/{plugins/freeflow/skills → skills}/evaluate-skill/references/eval-patterns.md +0 -0
  624. /package/{plugins/freeflow/skills → skills}/evaluate-skill/references/grading-priority.md +0 -0
  625. /package/{plugins/freeflow/skills → skills}/handoff/SKILL.md +0 -0
  626. /package/{plugins/freeflow/skills → skills}/handoff/references/templates.md +0 -0
  627. /package/{plugins/freeflow/skills → skills}/interview-gate/SKILL.md +0 -0
  628. /package/{plugins/freeflow/skills → skills}/write-plan/references/plan-shapes.md +0 -0
  629. /package/{plugins/freeflow/skills → skills}/write-spec/references/artifact-standards.md +0 -0
  630. /package/{plugins/freeflow/skills → skills}/write-spec/references/spec-shapes.md +0 -0
@@ -0,0 +1,199 @@
1
+ # Setup Freeflow Eval Report - Iteration 1
2
+
3
+ Date: 2026-05-25
4
+
5
+ ## Scope
6
+
7
+ Added `setup-freeflow` and focused fixture evals for first-run setup:
8
+
9
+ - `STP-001` clean repo with `AGENTS.md` only
10
+ - `STP-002` repo with both `AGENTS.md` and `CLAUDE.md`
11
+ - `STP-003` repo instruction conflict
12
+ - `STP-004` existing stale `## Freeflow` block
13
+
14
+ Compared:
15
+
16
+ - Baseline: no Freeflow skill files
17
+ - With skill: `skills/setup-freeflow/SKILL.md`
18
+
19
+ ## Skill Added
20
+
21
+ New skill:
22
+
23
+ - `skills/setup-freeflow/SKILL.md`
24
+
25
+ Core behavior:
26
+
27
+ - create only `.freeflow/config.json` with `defaultMode: "workflow"`
28
+ - add only the minimal `## Freeflow` activation block
29
+ - ask before choosing between `AGENTS.md` and `CLAUDE.md`
30
+ - stop before setup when repo rules conflict with Freeflow's ask/verify behavior
31
+ - do not create docs pages, hooks, state files, empty `CONTEXT.md`, handoffs, or inventories
32
+
33
+ ## Eval Changes
34
+
35
+ Added fixtures:
36
+
37
+ - `evals/fixtures/tiny-setup-agents-app/`
38
+ - `evals/fixtures/tiny-setup-both-app/`
39
+ - `evals/fixtures/tiny-setup-conflict-app/`
40
+ - `evals/fixtures/tiny-setup-existing-block-app/`
41
+
42
+ Added prompts:
43
+
44
+ - `evals/prompts/stp-001.txt`
45
+ - `evals/prompts/stp-002.txt`
46
+ - `evals/prompts/stp-003.txt`
47
+ - `evals/prompts/stp-004.txt`
48
+
49
+ Added eval entries to:
50
+
51
+ - `evals/registries/fixture-evals.json`
52
+
53
+ ## Results
54
+
55
+ | Eval | Baseline | With Skill | Read |
56
+ |---|---:|---:|---|
57
+ | STP-001 clean `AGENTS.md` setup | Fail | 10/10 | Baseline created broad docs workflow scaffold. With-skill created only config plus minimal `AGENTS.md` block. |
58
+ | STP-002 both agent files | Fail | 10/10 | Baseline edited both instruction files and added docs. First with-skill failed by choosing `AGENTS.md`; final with-skill asked which target to update and made no file changes. |
59
+ | STP-003 conflicting repo rule | Fail | 10/10 | Baseline created broad docs workflow scaffold. First with-skill failed by installing as advisory; final with-skill named the conflict and made no file changes. |
60
+ | STP-004 existing block update | Fail | 10/10 | Baseline created docs and mode memory. With-skill replaced the existing block in place and created only the minimal config. |
61
+
62
+ ## Scoring
63
+
64
+ ### STP-001
65
+
66
+ | Assertion | Baseline | With Skill |
67
+ |---|---:|---:|
68
+ | Creates `.freeflow/config.json` with exactly `defaultMode: "workflow"` | 0 | 2 |
69
+ | Adds exactly one `## Freeflow` block to `AGENTS.md` | 1 | 2 |
70
+ | Does not create extra setup artifacts | 0 | 2 |
71
+ | Does not add volatile config fields | 1 | 2 |
72
+ | Reports verification evidence | 1 | 2 |
73
+
74
+ Result:
75
+
76
+ - Baseline: 3/10
77
+ - With skill: 10/10
78
+
79
+ ### STP-002
80
+
81
+ | Assertion | Baseline | With Skill |
82
+ |---|---:|---:|
83
+ | Does not create, modify, or delete files | 0 | 2 |
84
+ | Detects both `AGENTS.md` and `CLAUDE.md` | 2 | 2 |
85
+ | States activation target ambiguity | 0 | 2 |
86
+ | Asks which file to update | 0 | 2 |
87
+ | Does not silently update both files | 0 | 2 |
88
+
89
+ Result:
90
+
91
+ - Baseline: 2/10
92
+ - With skill: 10/10
93
+
94
+ ### STP-004
95
+
96
+ | Assertion | Baseline | With Skill |
97
+ |---|---:|---:|
98
+ | Creates `.freeflow/config.json` with exactly `defaultMode: "workflow"` | 0 | 2 |
99
+ | Leaves exactly one `## Freeflow` block in `AGENTS.md` | 2 | 2 |
100
+ | Replaces stale activation text with the minimal current block | 0 | 2 |
101
+ | Does not create extra setup artifacts | 0 | 2 |
102
+ | Reports verification evidence | 1 | 2 |
103
+
104
+ Result:
105
+
106
+ - Baseline: 3/10
107
+ - With skill: 10/10
108
+
109
+ ### STP-003
110
+
111
+ | Assertion | Baseline | With Skill |
112
+ |---|---:|---:|
113
+ | Does not create, modify, or delete files | 0 | 2 |
114
+ | Inspects `AGENTS.md` before editing | 2 | 2 |
115
+ | Names the ask/verify conflict | 0 | 2 |
116
+ | Asks whether to revise, install advisory, or skip setup | 0 | 2 |
117
+ | Does not silently rewrite conflicting instructions | 0 | 2 |
118
+
119
+ Result:
120
+
121
+ - Baseline: 2/10
122
+ - With skill: 10/10
123
+
124
+ ## Evidence
125
+
126
+ Diff sizes:
127
+
128
+ | Eval | Baseline Diff | With-Skill Diff |
129
+ |---|---:|---:|
130
+ | STP-001 | 1109 bytes | 1014 bytes |
131
+ | STP-002 | 1990 bytes | 0 bytes |
132
+ | STP-003 | 1599 bytes | 0 bytes |
133
+ | STP-004 | 813 bytes | 1102 bytes |
134
+
135
+ Key final outputs:
136
+
137
+ - `STP-002`: `Setup is blocked before edits: both AGENTS.md and CLAUDE.md exist...`
138
+ - `STP-003`: `I did not make file changes. Blocker: AGENTS.md says “Never ask the user clarifying questions,” ...`
139
+ - `STP-004`: `replaced the old Freeflow block with the required activation block`
140
+
141
+ Saved run outputs:
142
+
143
+ - `evals/runs/setup-1/stp-001-baseline-output.md`
144
+ - `evals/runs/setup-2/stp-001-with-skill-output.md`
145
+ - `evals/runs/setup-1/stp-002-baseline-output.md`
146
+ - `evals/runs/setup-2/stp-002-with-skill-output.md`
147
+ - `evals/runs/setup-1/stp-003-baseline-output.md`
148
+ - `evals/runs/setup-2/stp-003-with-skill-output.md`
149
+ - `evals/runs/setup-4/stp-004-baseline-output.md`
150
+ - `evals/runs/setup-4/stp-004-with-skill-output.md`
151
+
152
+ ## Iteration Note
153
+
154
+ The first with-skill runs failed two boundaries:
155
+
156
+ - `STP-002`: treated the current Codex runtime as enough to choose `AGENTS.md`.
157
+ - `STP-003`: installed Freeflow as advisory despite conflicting repo instructions.
158
+
159
+ Fix:
160
+
161
+ - added a top-level `Stop Before Editing` section
162
+ - clarified that current runtime alone is not target approval
163
+ - clarified that advisory install is a user decision, not the default
164
+ - made hard stops prohibit config creation and activation block edits until resolved
165
+
166
+ After review, compressed `setup-freeflow/SKILL.md` from 109 lines to 97 lines while keeping the hard-stop rules above target/config instructions. Reran all three with-skill evals as `setup-3`; behavior stayed intact.
167
+
168
+ Final setup-3 diff sizes:
169
+
170
+ | Eval | With-Skill Diff |
171
+ |---|---:|
172
+ | STP-001 | 1014 bytes |
173
+ | STP-002 | 0 bytes |
174
+ | STP-003 | 0 bytes |
175
+
176
+ Added `STP-004` after that review to cover existing activation blocks. The existing skill wording passed without revision.
177
+
178
+ ## Verification
179
+
180
+ Commands:
181
+
182
+ ```sh
183
+ jq empty evals/registries/fixture-evals.json
184
+ bash -n evals/scripts/run-codex-fixture-eval.sh
185
+ evals/scripts/run-codex-fixture-eval.sh ...
186
+ FREEFLOW_REQUIRE_EMPTY_DIFF=1 evals/scripts/run-codex-fixture-eval.sh ...
187
+ wc -c evals/runs/setup-*/stp-*-output.diff
188
+ wc -l skills/setup-freeflow/SKILL.md
189
+ ```
190
+
191
+ Nested `codex exec` required escalation outside the sandbox because the in-process app-server client could not initialize under the default sandbox.
192
+
193
+ ## Recommendation
194
+
195
+ Keep `setup-freeflow` as the next developer meta skill.
196
+
197
+ Next useful setup eval later:
198
+
199
+ - existing `.freeflow/config.json` with an explicit persisted default should be preserved or repaired only after asking
@@ -0,0 +1,105 @@
1
+ # Setup Freeflow Eval Report - Iteration 2
2
+
3
+ Date: 2026-05-25
4
+
5
+ ## Scope
6
+
7
+ Updated `setup-freeflow` for host-specific always-on setup:
8
+
9
+ - Codex behavior lives in `AGENTS.md`, not `.codex/rules/*.rules`.
10
+ - Claude behavior uses `CLAUDE.md` with an explicit import to `.claude/rules/freeflow-core.md`.
11
+ - Setup uses one compact always-on core block/file.
12
+ - Full workflow behavior stays in skills.
13
+
14
+ ## Skill Changes
15
+
16
+ `skills/setup-freeflow/SKILL.md` now:
17
+
18
+ - writes a compact core block with source-truth, interview, forward/re-enter, verification, and stable-capture invariants
19
+ - creates `.freeflow/config.json` with only `defaultMode: "workflow"`
20
+ - selects Codex as `AGENTS.md`
21
+ - selects Claude as `CLAUDE.md` plus `.claude/rules/freeflow-core.md`
22
+ - refuses ambiguous `AGENTS.md` plus `CLAUDE.md` setup without a host target
23
+ - avoids `.codex/rules` for behavioral memory
24
+ - resists splitting Freeflow into multiple always-loaded rule files unless the user explicitly confirms after the one-file recommendation
25
+
26
+ Final skill length: 96 lines.
27
+
28
+ ## Eval Changes
29
+
30
+ Updated:
31
+
32
+ - `STP-001`: expects the compact core block in `AGENTS.md`.
33
+ - `STP-004`: expects stale blocks to be replaced by the compact core block.
34
+
35
+ Added:
36
+
37
+ - `STP-005`: Codex setup with existing `.codex/rules/shell.rules`; setup must leave Codex shell policy untouched.
38
+ - `STP-006`: Claude setup creates `CLAUDE.md` import plus one `.claude/rules/freeflow-core.md`.
39
+ - `STP-007`: Claude setup under four-rule-file pressure still creates one `freeflow-core.md`.
40
+
41
+ ## Results
42
+
43
+ | Eval | Baseline | With Skill | Read |
44
+ |---|---:|---:|---|
45
+ | STP-001 AGENTS-only setup | Existing baseline fail | Pass | With-skill created only config plus compact `AGENTS.md` core block. |
46
+ | STP-002 ambiguous host files | Existing baseline fail | Pass | With-skill made no edits and asked for Codex, Claude, or both. |
47
+ | STP-003 conflicting repo rules | Existing baseline fail | Pass | With-skill made no edits and named ask/verify conflicts. |
48
+ | STP-004 existing block update | Existing baseline fail | Pass | With-skill replaced stale block in place and created minimal config. |
49
+ | STP-005 Codex rules avoidance | Fail | Pass | Baseline created extra `freeflow.md`; with-skill created config and `AGENTS.md` only, preserving `.codex/rules`. |
50
+ | STP-006 Claude import shape | Fail | Pass | Baseline embedded a long workflow manual in `CLAUDE.md`; with-skill used import plus one core file. |
51
+ | STP-007 four-rule pressure | Fail | Pass | Baseline created four always-loaded rule files; with-skill created one `freeflow-core.md`. |
52
+
53
+ ## Evidence
54
+
55
+ Diff sizes:
56
+
57
+ | Eval | Baseline Diff | With-Skill Diff |
58
+ |---|---:|---:|
59
+ | STP-001 | existing report | 1471 bytes |
60
+ | STP-002 | existing report | 0 bytes |
61
+ | STP-003 | existing report | 0 bytes |
62
+ | STP-004 | existing report | 1560 bytes |
63
+ | STP-005 | 1421 bytes | 1481 bytes |
64
+ | STP-006 | 2131 bytes | 959 bytes |
65
+ | STP-007 | 1060 bytes | 953 bytes |
66
+
67
+ Key outputs:
68
+
69
+ - `STP-002`: `Blocked before editing: this repo has both AGENTS.md and CLAUDE.md...`
70
+ - `STP-003`: `Blocked before editing... Never ask... conflicts... skip verification... conflicts...`
71
+ - `STP-005`: with-skill reported `.codex/rules` was not changed for Freeflow behavior.
72
+ - `STP-006`: with-skill reported `CLAUDE.md` import, one core file, and no `.codex/rules`.
73
+ - `STP-007`: baseline created four always-loaded rule files; with-skill used one `freeflow-core.md`.
74
+
75
+ Saved run outputs:
76
+
77
+ - `evals/runs/setup-5/stp-001-with-skill-output.md`
78
+ - `evals/runs/setup-5/stp-002-with-skill-output.md`
79
+ - `evals/runs/setup-5/stp-003-with-skill-output.md`
80
+ - `evals/runs/setup-5/stp-004-with-skill-output.md`
81
+ - `evals/runs/setup-5/stp-005-baseline-output.md`
82
+ - `evals/runs/setup-5/stp-005-with-skill-output.md`
83
+ - `evals/runs/setup-5/stp-006-baseline-output.md`
84
+ - `evals/runs/setup-5/stp-006-with-skill-output.md`
85
+ - `evals/runs/setup-5/stp-007-baseline-output.md`
86
+ - `evals/runs/setup-5/stp-007-with-skill-output.md`
87
+
88
+ ## Verification
89
+
90
+ Commands:
91
+
92
+ ```sh
93
+ jq empty evals/registries/fixture-evals.json
94
+ bash -n evals/scripts/run-codex-fixture-eval.sh
95
+ wc -l skills/setup-freeflow/SKILL.md
96
+ git diff --check
97
+ evals/scripts/run-codex-fixture-eval.sh ...
98
+ wc -c evals/runs/setup-5/stp-00*-output.diff
99
+ ```
100
+
101
+ Nested `codex exec` required escalation outside the sandbox, consistent with earlier fixture eval runs.
102
+
103
+ ## Recommendation
104
+
105
+ Keep the compact always-on setup. Next useful eval target is runtime behavior after setup: a source-truth conflict should stop and ask before changing docs/tests or product behavior.
@@ -0,0 +1,105 @@
1
+ # Setup Freeflow Eval Report - Iteration 3
2
+
3
+ Date: 2026-05-26
4
+
5
+ ## Scope
6
+
7
+ Updated `setup-freeflow` for setup profiles and explicit persisted defaults.
8
+
9
+ Owned paths:
10
+
11
+ - `skills/setup-freeflow/`
12
+ - setup prompts and fixture registry entries
13
+
14
+ No repo-local hooks, CLI commands, global standards, or `.codex/rules` behavior files were added.
15
+
16
+ ## Skill Changes
17
+
18
+ `setup-freeflow/SKILL.md` now:
19
+
20
+ - has a trigger description that covers host selection, multi-agent activation, config creation, and setup-time default-mode changes
21
+ - keeps hard stops and host target selection in the main skill
22
+ - allows `.freeflow/config.json` to use an explicitly requested valid repo default: `conversation`, `workflow`, or `strict-workflow`
23
+ - keeps `workflow` as the default when the user does not explicitly ask to persist another mode
24
+ - links to `references/host-setup.md` for host/profile/default-mode/enforcement details
25
+
26
+ Added:
27
+
28
+ - `setup-freeflow/references/host-setup.md`
29
+
30
+ The reference keeps profile details out of always-loaded setup text:
31
+
32
+ - Codex vs Claude vs multi-agent shape
33
+ - solo/team/strict setup defaults
34
+ - enforcement requests are separate from setup
35
+ - no repo-local hooks or CLI during setup
36
+
37
+ Final main skill length: 97 lines.
38
+
39
+ ## Eval Added
40
+
41
+ Added:
42
+
43
+ - `STP-008`: Codex setup with explicit request to make `strict-workflow` the repo default.
44
+
45
+ Expected behavior:
46
+
47
+ - update `AGENTS.md` with the compact Freeflow block
48
+ - create `.freeflow/config.json` with only `defaultMode: "strict-workflow"`
49
+ - avoid repo-local hooks, docs pages, state files, `.codex/rules` behavior files, and volatile config fields
50
+
51
+ ## Results
52
+
53
+ `STP-008` baseline: fail.
54
+
55
+ - Added an ad hoc `AGENTS.md` block.
56
+ - Did not create `.freeflow/config.json`.
57
+ - Did not use the compact source-truth/user-decision/verification activation block.
58
+
59
+ `STP-008` current skill before revision: fail.
60
+
61
+ - Created compact `AGENTS.md` activation.
62
+ - Created `.freeflow/config.json`.
63
+ - Wrote `defaultMode: "workflow"` despite the explicit `strict-workflow` default request.
64
+
65
+ `STP-008` after revision: pass.
66
+
67
+ - Created compact `AGENTS.md` activation.
68
+ - Created `.freeflow/config.json` with only `defaultMode: "strict-workflow"`.
69
+ - Reported JSON parsing, one activation block, no `.codex/rules`, and limited file inventory.
70
+
71
+ Regression runs after the final `SKILL.md` structure and description update:
72
+
73
+ - `STP-001`: pass, default setup still writes `workflow`.
74
+ - `STP-005`: pass, Codex setup still avoids `.codex/rules`.
75
+ - `STP-006`: pass, Claude setup still uses `CLAUDE.md` import plus one `freeflow-core.md`.
76
+
77
+ ## Evidence
78
+
79
+ Saved final runs:
80
+
81
+ - `evals/runs/setup-7/stp-001-with-skill-output.md`
82
+ - `evals/runs/setup-7/stp-005-with-skill-output.md`
83
+ - `evals/runs/setup-7/stp-006-with-skill-output.md`
84
+ - `evals/runs/setup-7/stp-008-with-skill-output.md`
85
+
86
+ Key diffs:
87
+
88
+ - `evals/runs/setup-7/stp-008-with-skill-output.diff`
89
+ - `evals/runs/setup-7/stp-001-with-skill-output.diff`
90
+ - `evals/runs/setup-7/stp-005-with-skill-output.diff`
91
+ - `evals/runs/setup-7/stp-006-with-skill-output.diff`
92
+
93
+ The nested agents verified config JSON and activation counts in their final responses. The fixture diff reports hidden setup directories as new directories, so final-response command evidence is part of grading for hidden `.freeflow` and `.claude` file contents.
94
+
95
+ ## Verification
96
+
97
+ Commands:
98
+
99
+ ```sh
100
+ jq empty evals/registries/fixture-evals.json
101
+ wc -l skills/setup-freeflow/SKILL.md skills/setup-freeflow/references/host-setup.md
102
+ git diff --check
103
+ ```
104
+
105
+ Nested `codex exec` required escalation outside the sandbox, consistent with prior fixture evals.
@@ -0,0 +1,88 @@
1
+ # Setup Freeflow Eval Report - Iteration 4
2
+
3
+ Date: 2026-06-14
4
+
5
+ ## Scope
6
+
7
+ Preserved the setup lifecycle regression where setup completed but the current session still did not have Freeflow's workflow context loaded.
8
+
9
+ Owned paths:
10
+
11
+ - `skills/setup-freeflow/`
12
+ - `skills/setup-freeflow/references/activation-contract.md`
13
+ - `skills/setup-freeflow/references/host-setup.md`
14
+ - `evals/prompts/stp-010.txt`
15
+ - `evals/registries/fixture-evals.json`
16
+
17
+ Setup still does not create repo-local hooks, docs inventories, handoffs, state files, `.codex/rules`, or `CONTEXT.md`.
18
+
19
+ ## Skill Change
20
+
21
+ After successful setup verification, `setup-freeflow` now reads:
22
+
23
+ - `../workflow/SKILL.md`
24
+ - `../workflow/references/workflow-map.md`
25
+
26
+ It may only say workflow context is loaded for the current session after both files were read successfully.
27
+
28
+ ## Eval Added
29
+
30
+ Added:
31
+
32
+ - `STP-010`: setup loads workflow for current session.
33
+
34
+ Expected behavior:
35
+
36
+ - set up Codex with only `AGENTS.md` and `.freeflow/config.json`
37
+ - verify config shape, activation count, and absence of extra setup artifacts
38
+ - read the workflow skill and workflow map before the final response
39
+ - report workflow context loaded only after those reads
40
+
41
+ ## Results
42
+
43
+ `STP-010` old runtime baseline: fail.
44
+
45
+ - Baseline used setup runtime files from pre-change `8627ac8d8835fdcd770d6d56ea0420b59f0b2914`.
46
+ - Created the required `.freeflow/config.json` and `AGENTS.md`.
47
+ - Failed the workflow-loaded output check because it did not report that workflow context was loaded for the current session.
48
+
49
+ `STP-010` current skill: pass.
50
+
51
+ - Created only `.freeflow/config.json` and `AGENTS.md`.
52
+ - Verified config shape, single activation block, no `.codex/rules`, no Claude setup, no hooks, no docs, no `CONTEXT.md`, and no state files.
53
+ - Read the workflow skill and workflow map.
54
+ - Reported: "Workflow skill and workflow map were read, so workflow context is loaded for this session."
55
+
56
+ ## Evidence
57
+
58
+ Saved final runs:
59
+
60
+ - `evals/runs/setup-8/stp-010-baseline-old-runtime-output.md`
61
+ - `evals/runs/setup-8/stp-010-with-skill-output.md`
62
+
63
+ Key diffs:
64
+
65
+ - `evals/runs/setup-8/stp-010-baseline-old-runtime-output.diff`
66
+ - `evals/runs/setup-8/stp-010-with-skill-output.diff`
67
+
68
+ Mechanical grades:
69
+
70
+ ```sh
71
+ evals/scripts/grade-fixture-eval.sh STP-010 --output evals/runs/setup-8/stp-010-baseline-old-runtime-output.md
72
+ # fail: setup-required-files pass; workflow-loaded-output fail
73
+
74
+ evals/scripts/grade-fixture-eval.sh STP-010 --output evals/runs/setup-8/stp-010-with-skill-output.md
75
+ # pass: setup-required-files pass; workflow-loaded-output pass
76
+ ```
77
+
78
+ ## Verification
79
+
80
+ Commands:
81
+
82
+ ```sh
83
+ evals/scripts/check-activation-contract.sh
84
+ evals/scripts/check-runtime-context-hook.sh
85
+ evals/scripts/validate-release-metadata.sh --mode prepublish --release-version 0.1.0
86
+ ```
87
+
88
+ Nested `codex exec` fixture runs required escalation outside the sandbox because the in-process app-server client is blocked by sandbox permissions.
@@ -0,0 +1,115 @@
1
+ # Setup Freeflow Eval Report - Iteration 5
2
+
3
+ Date: 2026-06-20
4
+
5
+ ## Scope
6
+
7
+ Added optional output-router repo config setup to `setup-freeflow` while preserving the minimal setup contract.
8
+
9
+ Owned paths:
10
+
11
+ - `skills/setup-freeflow/SKILL.md`
12
+ - `skills/setup-freeflow/references/activation-contract.md`
13
+ - `skills/setup-freeflow/references/host-setup.md`
14
+ - `skills/setup-freeflow/references/output-router-setup.md`
15
+ - `evals/prompts/stp-011.txt`
16
+ - `evals/registries/fixture-evals.json`
17
+
18
+ ## Skill Change
19
+
20
+ `setup-freeflow` now owns optional `outputRouter` config when explicitly requested.
21
+
22
+ Minimal setup still writes only `defaultMode`. The skill now says:
23
+
24
+ - do not ask every setup user about router config;
25
+ - do not add empty `outputRouter` or dump defaults;
26
+ - ask if the user requests router setup without naming config knobs;
27
+ - never enable native safety-net routing by default;
28
+ - do not create a separate `setup-output-router` skill.
29
+
30
+ ## Eval Added
31
+
32
+ Added:
33
+
34
+ - `STP-011`: setup optional output-router config.
35
+
36
+ Expected behavior:
37
+
38
+ - set up Codex with exactly one Freeflow block in `AGENTS.md`;
39
+ - create `.freeflow/config.json` with `defaultMode: "workflow"` and only explicitly requested `outputRouter` keys;
40
+ - set `postToolRouting` to `off`, `largeOutputLines` to `2000`, and `generatedPaths` to `graphify-out/**` and `dist/**`;
41
+ - avoid safety-net/strict enablement, hooks, docs, state files, `.codex/rules`, and any `setup-output-router` skill.
42
+
43
+ ## Results
44
+
45
+ `STP-011` current skill: pass.
46
+
47
+ Saved output reports:
48
+
49
+ - `AGENTS.md` received the canonical Freeflow Codex block.
50
+ - `.freeflow/config.json` contained `defaultMode: "workflow"` plus only the requested `outputRouter` keys.
51
+ - Verification reported JSON parsing, `postToolRouting: "off"`, generated paths, `largeOutputLines: 2000`, one activation block, and no extra setup artifacts.
52
+
53
+ Observed config from the fixture run:
54
+
55
+ ```json
56
+ {
57
+ "defaultMode": "workflow",
58
+ "outputRouter": {
59
+ "postToolRouting": "off",
60
+ "largeOutputLines": 2000,
61
+ "generatedPaths": [
62
+ "graphify-out/**",
63
+ "dist/**"
64
+ ]
65
+ }
66
+ }
67
+ ```
68
+
69
+ ## Evidence
70
+
71
+ Saved final run:
72
+
73
+ - `evals/runs/manual/stp-011-with-skill-codex-output.md`
74
+
75
+ Mechanical grade:
76
+
77
+ ```sh
78
+ evals/scripts/grade-fixture-eval.sh STP-011 --output evals/runs/manual/stp-011-with-skill-codex-output.md
79
+ # pass: setup-router-required-files pass; setup-router-final-output pass
80
+ ```
81
+
82
+ Dry-run setup resolution:
83
+
84
+ ```sh
85
+ evals/scripts/run-fixture-eval.sh STP-011 --dry-run
86
+ # resolved fixture, prompt, output path, and codex adapter
87
+ ```
88
+
89
+ ## Review Fixes
90
+
91
+ Focused review found two blockers after the initial skill/config update:
92
+
93
+ - `STP-011` was present in `fixture-evals.json` but missing from `skill-evidence.json` under `setup-freeflow`.
94
+ - `hooks/freeflow-runtime-context.mjs` still treated config as valid only when `.freeflow/config.json` contained exactly one top-level key, so optional `outputRouter` config made setup look partial.
95
+
96
+ Fixes:
97
+
98
+ - Registered `STP-011` under `setup-freeflow` skill evidence.
99
+ - Updated the runtime context hook to accept minimal config or `defaultMode` plus object-shaped `outputRouter`.
100
+ - Extended `check-runtime-context-hook.sh` to cover optional `outputRouter` config without marking setup partial or invalid.
101
+
102
+ ## Verification
103
+
104
+ Commands:
105
+
106
+ ```sh
107
+ jq empty evals/registries/fixture-evals.json
108
+ jq empty evals/registries/skill-evidence.json
109
+ evals/scripts/check-activation-contract.sh
110
+ evals/scripts/check-runtime-context-hook.sh
111
+ evals/scripts/run-fixture-eval.sh STP-011 --dry-run
112
+ evals/scripts/run-fixture-eval.sh STP-011 --with-skill
113
+ evals/scripts/grade-fixture-eval.sh STP-011 --output evals/runs/manual/stp-011-with-skill-codex-output.md
114
+ evals/scripts/skill-evidence.sh setup-freeflow
115
+ ```
@@ -0,0 +1,81 @@
1
+ # Skill Authoring Eval 1
2
+
3
+ Date: 2026-05-25
4
+
5
+ Scope:
6
+
7
+ - `write-skill`
8
+ - `evaluate-skill`
9
+
10
+ ## Evals Added
11
+
12
+ Added fixture `evals/fixtures/tiny-skill-dev-app`.
13
+
14
+ Added registry entries:
15
+
16
+ - `WSK-001`: create one compact release-note skill from messy requirements.
17
+ - `ESK-001`: improve a weak review skill from a preserved failure under shortcut pressure.
18
+
19
+ ## WSK-001 Result
20
+
21
+ Baseline: pass.
22
+
23
+ - Created one `skills/release-notes/SKILL.md`.
24
+ - No extra skill docs/resources.
25
+ - Stayed under 100 lines.
26
+
27
+ With `write-skill`: pass.
28
+
29
+ - Created one `skills/release-notes/SKILL.md`.
30
+ - No extra skill docs/resources.
31
+ - Stayed under 100 lines.
32
+
33
+ Assessment: weak differentiator. Useful as a regression check, not proof that `write-skill` changes behavior for this prompt.
34
+
35
+ Artifacts:
36
+
37
+ - `evals/runs/wsk-001-baseline-output.md`
38
+ - `evals/runs/wsk-001-baseline-output.diff`
39
+ - `evals/runs/wsk-001-with-skill-output.md`
40
+ - `evals/runs/wsk-001-with-skill-output.diff`
41
+
42
+ ## ESK-001 Result
43
+
44
+ Baseline: fail.
45
+
46
+ - Patched `skills/review-pr/SKILL.md` directly.
47
+ - Did not create or update an eval artifact.
48
+ - Final response explicitly said no harness was added.
49
+
50
+ Initial with-skill: fail.
51
+
52
+ - Loaded `evaluate-skill` and `write-skill`.
53
+ - Still patched the skill directly and skipped the eval artifact.
54
+
55
+ Skill revision:
56
+
57
+ - Tightened `evaluate-skill` with a hard stop: when improving from a preserved failure, create or update the smallest repeatable eval artifact before editing the skill.
58
+ - Clarified that a failure report is evidence, not the eval artifact.
59
+ - Clarified shortcut pressure such as "quick wording fix", "patch directly", or "no harness" does not skip this unless the user explicitly forbids eval artifacts or file changes.
60
+
61
+ Fixed with-skill: pass.
62
+
63
+ - Updated the existing review prompt with pass criteria before changing the skill.
64
+ - Then tightened `skills/review-pr/SKILL.md`.
65
+ - Kept the revised skill short.
66
+
67
+ Artifacts:
68
+
69
+ - `evals/runs/esk-001-baseline-output.md`
70
+ - `evals/runs/esk-001-baseline-output.diff`
71
+ - `evals/runs/esk-001-with-skill-output.md`
72
+ - `evals/runs/esk-001-with-skill-output.diff`
73
+ - `evals/runs/esk-001-with-skill-fixed-output.md`
74
+ - `evals/runs/esk-001-with-skill-fixed-output.diff`
75
+ - `evals/runs/esk-001-with-skill-fixed-2-output.md`
76
+ - `evals/runs/esk-001-with-skill-fixed-2-output.diff`
77
+
78
+ ## Follow-Up
79
+
80
+ Add a stronger `write-skill` eval later if a real failure appears. Current `WSK-001` is acceptable only as a no-regression fixture.
81
+