@hassangameryt/freeflow 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (692) hide show
  1. package/.claude-plugin/marketplace.json +4 -4
  2. package/.claude-plugin/plugin.json +6 -2
  3. package/.codex-plugin/plugin.json +9 -9
  4. package/README.md +190 -127
  5. package/assets/feedback-loop.png +0 -0
  6. package/assets/freeflow_workflow_map.png +0 -0
  7. package/assets/workflow.png +0 -0
  8. package/{skills → capabilities}/output-router/SKILL.md +21 -17
  9. package/{skills → capabilities}/output-router/references/safety-policy.md +1 -1
  10. package/capabilities/output-router/references/setup.md +110 -0
  11. package/command-surface.json +62 -7
  12. package/hooks/freeflow-runtime-context.mjs +572 -154
  13. package/hooks/hooks.json +26 -1
  14. package/hooks/tests/check-runtime-context-hook.sh +375 -0
  15. package/package.json +24 -24
  16. package/pi-extension/dist/index.js +264 -80
  17. package/pi-extension/dist/integrations/mcp-config.js +29 -0
  18. package/pi-extension/dist/output-router/host-producer-identification.js +232 -0
  19. package/pi-extension/dist/output-router/native-safety-net.js +216 -0
  20. package/pi-extension/dist/output-router/observed-tool-routing.js +122 -0
  21. package/pi-extension/dist/output-router/renderers.js +1023 -0
  22. package/pi-extension/dist/output-router/router-tools.js +554 -0
  23. package/pi-extension/dist/output-router/schemas.js +383 -0
  24. package/pi-extension/dist/output-router/status.js +744 -0
  25. package/pi-extension/dist/output-router/utils.js +502 -0
  26. package/pi-extension/dist/runtime/runtime-context.js +798 -0
  27. package/pi-extension/dist/settings/settings-tui.js +314 -0
  28. package/pi-extension/dist/settings/settings-ui.js +1487 -0
  29. package/pi-extension/freeflow/index.js +1 -0
  30. package/pi-extension/src/index.ts +230 -56
  31. package/pi-extension/src/{host-producer-identification.ts → output-router/host-producer-identification.ts} +37 -7
  32. package/pi-extension/src/{native-safety-net.ts → output-router/native-safety-net.ts} +8 -12
  33. package/pi-extension/src/{observed-tool-routing.ts → output-router/observed-tool-routing.ts} +19 -11
  34. package/pi-extension/src/{renderers.ts → output-router/renderers.ts} +246 -55
  35. package/pi-extension/src/{router-tools.ts → output-router/router-tools.ts} +130 -29
  36. package/pi-extension/src/{schemas.ts → output-router/schemas.ts} +171 -43
  37. package/pi-extension/src/output-router/status.ts +791 -0
  38. package/pi-extension/src/{utils.ts → output-router/utils.ts} +80 -18
  39. package/pi-extension/src/runtime/runtime-context.ts +895 -0
  40. package/pi-extension/src/settings/settings-tui.ts +408 -0
  41. package/pi-extension/src/settings/settings-ui.ts +1730 -0
  42. package/pi-extension/tests/integration/pi-extension.test.js +4020 -0
  43. package/pi-extension/tests/output-router/observed-producers.test.js +132 -0
  44. package/pi-extension/tests/output-router/search.test.js +722 -0
  45. package/pi-extension/tests/settings/settings-tui.test.js +243 -0
  46. package/router/dist/benchmarks/benchmark-harness.d.ts +16 -13
  47. package/router/dist/benchmarks/benchmark-harness.js +62 -62
  48. package/router/dist/benchmarks/benchmarks.d.ts +80 -73
  49. package/router/dist/benchmarks/benchmarks.js +860 -761
  50. package/router/dist/benchmarks/codex-qa-benchmarks.d.ts +71 -67
  51. package/router/dist/benchmarks/codex-qa-benchmarks.js +429 -390
  52. package/router/dist/benchmarks/command-benchmarks.d.ts +107 -92
  53. package/router/dist/benchmarks/command-benchmarks.js +616 -571
  54. package/router/dist/benchmarks/context-mode-normalized-benchmarks.d.ts +64 -56
  55. package/router/dist/benchmarks/context-mode-normalized-benchmarks.js +679 -613
  56. package/router/dist/benchmarks/context-mode-real-deep-benchmark.d.ts +103 -95
  57. package/router/dist/benchmarks/context-mode-real-deep-benchmark.js +1188 -1103
  58. package/router/dist/benchmarks/context-saving-baseline.d.ts +47 -41
  59. package/router/dist/benchmarks/context-saving-baseline.js +405 -374
  60. package/router/dist/benchmarks/index-benchmarks.d.ts +74 -69
  61. package/router/dist/benchmarks/index-benchmarks.js +761 -661
  62. package/router/dist/benchmarks/storage-policy-benchmarks.d.ts +80 -69
  63. package/router/dist/benchmarks/storage-policy-benchmarks.js +549 -469
  64. package/router/dist/benchmarks/vault-index-benchmarks.d.ts +43 -34
  65. package/router/dist/benchmarks/vault-index-benchmarks.js +294 -229
  66. package/router/dist/config/config.d.ts +62 -55
  67. package/router/dist/config/config.js +552 -373
  68. package/router/dist/config/router-contract.d.ts +2 -2
  69. package/router/dist/config/router-contract.js +38 -38
  70. package/router/dist/config/schema.d.ts +19 -10
  71. package/router/dist/config/schema.js +586 -538
  72. package/router/dist/config/types.d.ts +413 -361
  73. package/router/dist/config/types.js +19 -8
  74. package/router/dist/evidence/bounded-evidence.d.ts +26 -26
  75. package/router/dist/evidence/bounded-evidence.js +265 -252
  76. package/router/dist/evidence/evidence-range-selector.d.ts +16 -14
  77. package/router/dist/evidence/evidence-range-selector.js +192 -191
  78. package/router/dist/evidence/evidence-search.d.ts +18 -16
  79. package/router/dist/evidence/evidence-search.js +401 -377
  80. package/router/dist/evidence/evidence.d.ts +17 -17
  81. package/router/dist/evidence/evidence.js +163 -159
  82. package/router/dist/evidence/failure-contracts.d.ts +25 -15
  83. package/router/dist/evidence/failure-contracts.js +132 -126
  84. package/router/dist/evidence/line-ranges.d.ts +17 -15
  85. package/router/dist/evidence/line-ranges.js +18 -18
  86. package/router/dist/experiments/local-index.d.ts +47 -41
  87. package/router/dist/experiments/local-index.js +477 -458
  88. package/router/dist/local/local-traversal.d.ts +18 -0
  89. package/router/dist/local/local-traversal.js +334 -0
  90. package/router/dist/processing/engine.d.ts +94 -70
  91. package/router/dist/processing/engine.js +462 -364
  92. package/router/dist/processing/reducers.d.ts +208 -194
  93. package/router/dist/processing/reducers.js +1434 -1256
  94. package/router/dist/processing/renderers.d.ts +24 -18
  95. package/router/dist/processing/renderers.js +161 -152
  96. package/router/dist/processing/scripts.d.ts +80 -67
  97. package/router/dist/processing/scripts.js +343 -290
  98. package/router/dist/repo/repo-traversal.d.ts +9 -9
  99. package/router/dist/repo/repo-traversal.js +205 -202
  100. package/router/dist/routing/observed-reducers.d.ts +25 -19
  101. package/router/dist/routing/observed-reducers.js +559 -545
  102. package/router/dist/routing/observed-routing.d.ts +23 -14
  103. package/router/dist/routing/observed-routing.js +319 -318
  104. package/router/dist/routing/parsers.d.ts +10 -10
  105. package/router/dist/routing/parsers.js +454 -406
  106. package/router/dist/routing/run-filters.d.ts +45 -41
  107. package/router/dist/routing/run-filters.js +215 -207
  108. package/router/dist/routing/run-reducers.d.ts +45 -26
  109. package/router/dist/routing/run-reducers.js +102 -90
  110. package/router/dist/sandbox/adapter-roots.d.ts +34 -26
  111. package/router/dist/sandbox/adapter-roots.js +43 -41
  112. package/router/dist/sandbox/eryx-python-adapter.d.ts +9 -5
  113. package/router/dist/sandbox/eryx-python-adapter.js +659 -588
  114. package/router/dist/sandbox/jq-wasm-adapter.d.ts +3 -3
  115. package/router/dist/sandbox/jq-wasm-adapter.js +319 -311
  116. package/router/dist/sandbox/quickjs-wasi-adapter.d.ts +9 -5
  117. package/router/dist/sandbox/quickjs-wasi-adapter.js +336 -333
  118. package/router/dist/sandbox/script-sandbox.d.ts +105 -84
  119. package/router/dist/sandbox/script-sandbox.js +294 -260
  120. package/router/dist/setup/script-transform-adapters.d.ts +23 -19
  121. package/router/dist/setup/script-transform-adapters.js +223 -204
  122. package/router/dist/tools/batch.d.ts +27 -16
  123. package/router/dist/tools/batch.js +493 -416
  124. package/router/dist/tools/run.d.ts +47 -35
  125. package/router/dist/tools/run.js +1172 -1087
  126. package/router/dist/tools/search.d.ts +53 -31
  127. package/router/dist/tools/search.js +1224 -1065
  128. package/router/dist/transform/engine.d.ts +121 -96
  129. package/router/dist/transform/engine.js +1853 -1736
  130. package/router/dist/vault/vault-index.d.ts +97 -78
  131. package/router/dist/vault/vault-index.js +347 -342
  132. package/router/dist/vault/vault.d.ts +109 -74
  133. package/router/dist/vault/vault.js +528 -533
  134. package/runtime/interaction-contract.md +16 -0
  135. package/skills/bypass/SKILL.md +52 -38
  136. package/skills/commit-work/SKILL.md +54 -46
  137. package/skills/commit-work/references/staging-decisions.md +46 -24
  138. package/skills/decision-gate/SKILL.md +48 -0
  139. package/skills/design-for-depth/SKILL.md +75 -59
  140. package/skills/design-for-depth/references/design-pressure-signals.md +56 -7
  141. package/skills/design-for-depth/references/interface-design-loop.md +156 -0
  142. package/skills/design-for-depth/references/software-design-philosophy.md +32 -3
  143. package/skills/diagnose-failure/SKILL.md +65 -60
  144. package/skills/diagnose-failure/references/diagnostic-loop-catalog.md +36 -0
  145. package/skills/diagnose-failure/references/flaky-and-performance.md +30 -18
  146. package/skills/discuss/SKILL.md +121 -0
  147. package/skills/discuss/references/checkpoints.md +66 -0
  148. package/skills/evaluate-skill/SKILL.md +62 -53
  149. package/skills/evaluate-skill/references/definition-schema.md +345 -0
  150. package/skills/evaluate-skill/references/evaluation-design.md +86 -0
  151. package/skills/evaluate-skill/references/execution-and-evidence.md +85 -0
  152. package/skills/evaluate-skill/references/review-and-revision.md +51 -0
  153. package/skills/evaluate-skill/scripts/lib/definitions.mjs +296 -0
  154. package/skills/evaluate-skill/scripts/lib/evidence.mjs +46 -0
  155. package/skills/evaluate-skill/scripts/lib/grade-comparison.mjs +60 -0
  156. package/skills/evaluate-skill/scripts/lib/grade-interaction.mjs +148 -0
  157. package/skills/evaluate-skill/scripts/lib/grade-support.mjs +126 -0
  158. package/skills/evaluate-skill/scripts/lib/grade-workspace.mjs +192 -0
  159. package/skills/evaluate-skill/scripts/lib/grade.mjs +56 -0
  160. package/skills/evaluate-skill/scripts/lib/pi.mjs +489 -0
  161. package/skills/evaluate-skill/scripts/lib/process.mjs +79 -0
  162. package/skills/evaluate-skill/scripts/lib/rpc.mjs +444 -0
  163. package/skills/evaluate-skill/scripts/lib/runner.mjs +346 -0
  164. package/skills/evaluate-skill/scripts/lib/sandbox.mjs +409 -0
  165. package/skills/evaluate-skill/scripts/lib/view.mjs +337 -0
  166. package/skills/evaluate-skill/scripts/pi-guard.mjs +164 -0
  167. package/skills/evaluate-skill/scripts/skill-eval.mjs +101 -0
  168. package/skills/execute-work/SKILL.md +99 -0
  169. package/skills/execute-work/references/code-practices.md +78 -0
  170. package/skills/execute-work/references/execution-loop.md +77 -0
  171. package/skills/finish-branch/SKILL.md +117 -0
  172. package/skills/finish-branch/references/integration-options.md +80 -0
  173. package/skills/handoff/SKILL.md +65 -39
  174. package/skills/handoff/references/templates.md +104 -27
  175. package/skills/launch-work/SKILL.md +116 -0
  176. package/skills/launch-work/references/launch-readiness.md +85 -0
  177. package/skills/migration-work/SKILL.md +115 -0
  178. package/skills/migration-work/references/migration-lifecycle.md +82 -0
  179. package/skills/mode-contract/SKILL.md +72 -52
  180. package/skills/release-work/SKILL.md +118 -0
  181. package/skills/release-work/references/release-evidence.md +78 -0
  182. package/skills/review-artifact/SKILL.md +77 -69
  183. package/skills/review-artifact/references/reviewer-prompt.md +133 -42
  184. package/skills/review-work/SKILL.md +82 -57
  185. package/skills/review-work/references/reviewer-prompt.md +97 -60
  186. package/skills/review-work/references/security-risk-lens.md +67 -0
  187. package/skills/setup-freeflow/SKILL.md +54 -68
  188. package/skills/setup-freeflow/references/activation-contract.md +84 -69
  189. package/skills/setup-freeflow/references/host-setup.md +50 -17
  190. package/skills/simplify-code/SKILL.md +80 -0
  191. package/skills/simplify-code/references/simplification-patterns.md +70 -0
  192. package/skills/tdd/SKILL.md +96 -0
  193. package/skills/tdd/references/test-design.md +96 -0
  194. package/skills/track-work/SKILL.md +84 -0
  195. package/skills/track-work/references/working-record-schema.md +378 -0
  196. package/skills/verify-work/SKILL.md +59 -38
  197. package/skills/verify-work/references/browser-runtime-evidence.md +55 -0
  198. package/skills/verify-work/references/integration-evidence.md +51 -0
  199. package/skills/verify-work/references/performance-evidence.md +57 -0
  200. package/skills/workflow/SKILL.md +53 -59
  201. package/skills/workflow/references/domain-skill-composition.md +57 -0
  202. package/skills/workflow/references/workflow-loop.md +92 -0
  203. package/skills/write-plan/SKILL.md +85 -58
  204. package/skills/write-plan/references/plan-shapes.md +132 -46
  205. package/skills/write-skill/SKILL.md +82 -51
  206. package/skills/write-skill/references/activation-boundaries.md +62 -0
  207. package/skills/write-skill/references/agent-first-instructions.md +87 -0
  208. package/skills/write-skill/references/progressive-disclosure.md +63 -0
  209. package/skills/write-skill/scripts/lib/skill-author.mjs +653 -0
  210. package/skills/write-skill/scripts/skill-author.mjs +104 -0
  211. package/skills/write-spec/SKILL.md +65 -54
  212. package/skills/write-spec/references/artifact-standards.md +60 -41
  213. package/skills/write-spec/references/decision-records.md +77 -0
  214. package/skills/write-spec/references/spec-shapes.md +119 -60
  215. package/evals/README.md +0 -32
  216. package/evals/fixtures/copy-conventions.md +0 -31
  217. package/evals/fixtures/output-router/README.md +0 -12
  218. package/evals/fixtures/output-router/failed-command-output.txt +0 -9
  219. package/evals/fixtures/output-router/large-router-manual.md +0 -63
  220. package/evals/fixtures/output-router/native-large-output.txt +0 -40
  221. package/evals/fixtures/output-router/noisy-test-output.txt +0 -30
  222. package/evals/fixtures/output-router/verification-output.txt +0 -8
  223. package/evals/fixtures/tiny-artifact-review-app/README.md +0 -5
  224. package/evals/fixtures/tiny-artifact-review-app/docs/billing-policy.md +0 -10
  225. package/evals/fixtures/tiny-artifact-review-app/docs/specs/billing-webhook-api-draft.md +0 -21
  226. package/evals/fixtures/tiny-artifact-review-app/docs/specs/failed-payment-grace-period.md +0 -24
  227. package/evals/fixtures/tiny-artifact-review-app/docs/specs/immediate-downgrade.md +0 -17
  228. package/evals/fixtures/tiny-artifact-review-app/tests/billing.test.md +0 -8
  229. package/evals/fixtures/tiny-commit-ambiguous-app/.freeflow-eval-setup.sh +0 -15
  230. package/evals/fixtures/tiny-commit-ambiguous-app/README.md +0 -3
  231. package/evals/fixtures/tiny-commit-ambiguous-app/src/billing.ts +0 -5
  232. package/evals/fixtures/tiny-commit-ambiguous-app/src/settings-page.ts +0 -1
  233. package/evals/fixtures/tiny-commit-ambiguous-app/tests/settings.test.md +0 -5
  234. package/evals/fixtures/tiny-commit-app/.freeflow-eval-setup.sh +0 -14
  235. package/evals/fixtures/tiny-commit-app/README.md +0 -3
  236. package/evals/fixtures/tiny-commit-app/docs/specs/settings-title.md +0 -5
  237. package/evals/fixtures/tiny-commit-app/src/billing.ts +0 -5
  238. package/evals/fixtures/tiny-commit-app/src/settings-page.ts +0 -1
  239. package/evals/fixtures/tiny-commit-app/tests/settings.test.md +0 -5
  240. package/evals/fixtures/tiny-commit-everything-app/.freeflow-eval-setup.sh +0 -15
  241. package/evals/fixtures/tiny-commit-everything-app/README.md +0 -3
  242. package/evals/fixtures/tiny-commit-everything-app/docs/specs/settings-title.md +0 -5
  243. package/evals/fixtures/tiny-commit-everything-app/src/billing.ts +0 -5
  244. package/evals/fixtures/tiny-commit-everything-app/src/settings-page.ts +0 -1
  245. package/evals/fixtures/tiny-commit-everything-app/tests/settings.test.md +0 -5
  246. package/evals/fixtures/tiny-commit-staged-app/.freeflow-eval-setup.sh +0 -15
  247. package/evals/fixtures/tiny-commit-staged-app/README.md +0 -3
  248. package/evals/fixtures/tiny-commit-staged-app/docs/specs/settings-title.md +0 -5
  249. package/evals/fixtures/tiny-commit-staged-app/src/billing.ts +0 -5
  250. package/evals/fixtures/tiny-commit-staged-app/src/settings-page.ts +0 -1
  251. package/evals/fixtures/tiny-commit-staged-app/tests/settings.test.md +0 -5
  252. package/evals/fixtures/tiny-commit-staged-mixed-app/.freeflow-eval-setup.sh +0 -15
  253. package/evals/fixtures/tiny-commit-staged-mixed-app/README.md +0 -3
  254. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/billing-policy.md +0 -3
  255. package/evals/fixtures/tiny-commit-staged-mixed-app/docs/release-notes.md +0 -3
  256. package/evals/fixtures/tiny-commit-staged-mixed-app/src/billing.ts +0 -5
  257. package/evals/fixtures/tiny-dashboard-stale-app/README.md +0 -11
  258. package/evals/fixtures/tiny-dashboard-stale-app/src/api-client.ts +0 -13
  259. package/evals/fixtures/tiny-dashboard-stale-app/src/dashboard-data.ts +0 -19
  260. package/evals/fixtures/tiny-dashboard-stale-app/tests/dashboard-data.test.md +0 -9
  261. package/evals/fixtures/tiny-decision-destination-app/README.md +0 -6
  262. package/evals/fixtures/tiny-decision-destination-app/src/risk.ts +0 -8
  263. package/evals/fixtures/tiny-decision-destination-app/tests/risk.test.md +0 -4
  264. package/evals/fixtures/tiny-design-depth-app/README.md +0 -15
  265. package/evals/fixtures/tiny-design-depth-app/src/routes.ts +0 -25
  266. package/evals/fixtures/tiny-design-depth-app/src/sendEmail.ts +0 -3
  267. package/evals/fixtures/tiny-design-depth-app/src/sendSms.ts +0 -3
  268. package/evals/fixtures/tiny-design-depth-app/src/telemetry.ts +0 -3
  269. package/evals/fixtures/tiny-design-depth-app/tests/routes.test.md +0 -12
  270. package/evals/fixtures/tiny-eval-method-app/README.md +0 -8
  271. package/evals/fixtures/tiny-eval-method-app/evals/reports/review-pr-eval-result.md +0 -5
  272. package/evals/fixtures/tiny-eval-method-app/evals/reports/triage-question-failure.md +0 -17
  273. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.diff +0 -11
  274. package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.md +0 -3
  275. package/evals/fixtures/tiny-eval-method-app/skills/review-pr/SKILL.md +0 -10
  276. package/evals/fixtures/tiny-eval-method-app/skills/triage-question/SKILL.md +0 -10
  277. package/evals/fixtures/tiny-execute-plan-app/README.md +0 -5
  278. package/evals/fixtures/tiny-execute-plan-app/checks/settings-and-billing-check.sh +0 -5
  279. package/evals/fixtures/tiny-execute-plan-app/docs/billing-policy.md +0 -10
  280. package/evals/fixtures/tiny-execute-plan-app/docs/plans/immediate-downgrade-plan.md +0 -11
  281. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-plan.md +0 -12
  282. package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-with-bad-check.md +0 -15
  283. package/evals/fixtures/tiny-execute-plan-app/docs/plans/unverified-billing-plan.md +0 -9
  284. package/evals/fixtures/tiny-execute-plan-app/docs/specs/failed-payment-grace-period.md +0 -9
  285. package/evals/fixtures/tiny-execute-plan-app/docs/specs/settings-title.md +0 -9
  286. package/evals/fixtures/tiny-execute-plan-app/src/billing.ts +0 -7
  287. package/evals/fixtures/tiny-execute-plan-app/src/settings-page.ts +0 -1
  288. package/evals/fixtures/tiny-execute-plan-app/tests/billing.test.md +0 -7
  289. package/evals/fixtures/tiny-execute-plan-app/tests/settings.test.md +0 -5
  290. package/evals/fixtures/tiny-execute-review-loop-app/README.md +0 -5
  291. package/evals/fixtures/tiny-execute-review-loop-app/docs/billing-policy.md +0 -9
  292. package/evals/fixtures/tiny-execute-review-loop-app/docs/plans/settings-title-plan.md +0 -12
  293. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/previous-agent-summary.md +0 -15
  294. package/evals/fixtures/tiny-execute-review-loop-app/docs/review/review-pass-1-non-pass.md +0 -11
  295. package/evals/fixtures/tiny-execute-review-loop-app/docs/specs/settings-title.md +0 -9
  296. package/evals/fixtures/tiny-execute-review-loop-app/src/billing.ts +0 -7
  297. package/evals/fixtures/tiny-execute-review-loop-app/src/settings-page.ts +0 -1
  298. package/evals/fixtures/tiny-execute-review-loop-app/tests/billing.test.md +0 -7
  299. package/evals/fixtures/tiny-execute-review-loop-app/tests/settings.test.md +0 -5
  300. package/evals/fixtures/tiny-flaky-session-app/README.md +0 -10
  301. package/evals/fixtures/tiny-flaky-session-app/src/session-timeout.ts +0 -13
  302. package/evals/fixtures/tiny-flaky-session-app/tests/session-timeout.test.md +0 -8
  303. package/evals/fixtures/tiny-handoff-app/README.md +0 -5
  304. package/evals/fixtures/tiny-handoff-app/docs/billing-policy.md +0 -10
  305. package/evals/fixtures/tiny-handoff-app/docs/handoffs/old-agent-handoff.md +0 -8
  306. package/evals/fixtures/tiny-handoff-app/docs/plans/billing-retry-plan.md +0 -11
  307. package/evals/fixtures/tiny-handoff-app/src/billing.ts +0 -7
  308. package/evals/fixtures/tiny-handoff-report-app/README.md +0 -3
  309. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.diff +0 -1
  310. package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.md +0 -1
  311. package/evals/fixtures/tiny-handoff-report-app/evals/scores/hof-003-score.txt +0 -4
  312. package/evals/fixtures/tiny-plan-app/README.md +0 -5
  313. package/evals/fixtures/tiny-plan-app/docs/billing-policy.md +0 -10
  314. package/evals/fixtures/tiny-plan-app/docs/specs/failed-payment-grace-period.md +0 -25
  315. package/evals/fixtures/tiny-plan-app/src/billing.ts +0 -17
  316. package/evals/fixtures/tiny-plan-app/src/settings-page.ts +0 -1
  317. package/evals/fixtures/tiny-plan-app/tests/billing.test.md +0 -8
  318. package/evals/fixtures/tiny-plan-app/tests/settings.test.md +0 -5
  319. package/evals/fixtures/tiny-plan-strict-api-app/README.md +0 -3
  320. package/evals/fixtures/tiny-plan-strict-api-app/docs/billing-policy.md +0 -5
  321. package/evals/fixtures/tiny-plan-strict-api-app/docs/specs/billing-webhook-api.md +0 -25
  322. package/evals/fixtures/tiny-plan-strict-api-app/src/billing.ts +0 -5
  323. package/evals/fixtures/tiny-plan-strict-api-app/src/webhooks.ts +0 -3
  324. package/evals/fixtures/tiny-plan-strict-api-app/tests/billing-webhook.test.md +0 -8
  325. package/evals/fixtures/tiny-post-setup-invalid-config-app/.freeflow/config.json +0 -3
  326. package/evals/fixtures/tiny-post-setup-invalid-config-app/AGENTS.md +0 -15
  327. package/evals/fixtures/tiny-post-setup-invalid-config-app/README.md +0 -16
  328. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/auth-review.md +0 -15
  329. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/billing-policy.md +0 -10
  330. package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/copy-conventions.md +0 -15
  331. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/auth-middleware.ts +0 -12
  332. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/billing.ts +0 -20
  333. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/login-copy.ts +0 -1
  334. package/evals/fixtures/tiny-post-setup-invalid-config-app/src/settings-page.ts +0 -1
  335. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/auth-middleware.test.md +0 -8
  336. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/billing.test.md +0 -8
  337. package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/copy.test.md +0 -7
  338. package/evals/fixtures/tiny-post-setup-missing-config-app/AGENTS.md +0 -15
  339. package/evals/fixtures/tiny-post-setup-missing-config-app/README.md +0 -16
  340. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/auth-review.md +0 -15
  341. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/billing-policy.md +0 -10
  342. package/evals/fixtures/tiny-post-setup-missing-config-app/docs/copy-conventions.md +0 -15
  343. package/evals/fixtures/tiny-post-setup-missing-config-app/src/auth-middleware.ts +0 -12
  344. package/evals/fixtures/tiny-post-setup-missing-config-app/src/billing.ts +0 -20
  345. package/evals/fixtures/tiny-post-setup-missing-config-app/src/login-copy.ts +0 -1
  346. package/evals/fixtures/tiny-post-setup-missing-config-app/src/settings-page.ts +0 -1
  347. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/auth-middleware.test.md +0 -8
  348. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/billing.test.md +0 -8
  349. package/evals/fixtures/tiny-post-setup-missing-config-app/tests/copy.test.md +0 -7
  350. package/evals/fixtures/tiny-post-setup-source-conflict-app/.freeflow/config.json +0 -3
  351. package/evals/fixtures/tiny-post-setup-source-conflict-app/AGENTS.md +0 -15
  352. package/evals/fixtures/tiny-post-setup-source-conflict-app/README.md +0 -16
  353. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/auth-review.md +0 -15
  354. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/billing-policy.md +0 -10
  355. package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/copy-conventions.md +0 -15
  356. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/auth-middleware.ts +0 -12
  357. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/billing.ts +0 -20
  358. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/login-copy.ts +0 -1
  359. package/evals/fixtures/tiny-post-setup-source-conflict-app/src/settings-page.ts +0 -1
  360. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/auth-middleware.test.md +0 -8
  361. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/billing.test.md +0 -8
  362. package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/copy.test.md +0 -7
  363. package/evals/fixtures/tiny-report-performance-app/README.md +0 -11
  364. package/evals/fixtures/tiny-report-performance-app/docs/performance.md +0 -10
  365. package/evals/fixtures/tiny-report-performance-app/src/report-summary.ts +0 -29
  366. package/evals/fixtures/tiny-report-performance-app/tests/report-performance.test.md +0 -8
  367. package/evals/fixtures/tiny-review-request-app/README.md +0 -3
  368. package/evals/fixtures/tiny-review-request-app/docs/billing-policy.md +0 -5
  369. package/evals/fixtures/tiny-review-request-app/docs/plans/billing-webhook-api.md +0 -19
  370. package/evals/fixtures/tiny-review-request-app/docs/review/previous-agent-summary.md +0 -20
  371. package/evals/fixtures/tiny-review-request-app/docs/specs/billing-webhook-api.md +0 -25
  372. package/evals/fixtures/tiny-review-request-app/src/billing.ts +0 -11
  373. package/evals/fixtures/tiny-review-request-app/src/webhooks.ts +0 -19
  374. package/evals/fixtures/tiny-review-request-app/tests/billing-webhook.test.md +0 -8
  375. package/evals/fixtures/tiny-review-work-app/README.md +0 -5
  376. package/evals/fixtures/tiny-review-work-app/docs/billing-policy.md +0 -9
  377. package/evals/fixtures/tiny-review-work-app/docs/plans/settings-title-plan.md +0 -12
  378. package/evals/fixtures/tiny-review-work-app/docs/review/previous-agent-summary.md +0 -15
  379. package/evals/fixtures/tiny-review-work-app/docs/specs/settings-title.md +0 -9
  380. package/evals/fixtures/tiny-review-work-app/src/billing.ts +0 -7
  381. package/evals/fixtures/tiny-review-work-app/src/settings-page.ts +0 -1
  382. package/evals/fixtures/tiny-review-work-app/tests/billing.test.md +0 -7
  383. package/evals/fixtures/tiny-review-work-app/tests/settings.test.md +0 -5
  384. package/evals/fixtures/tiny-saas-app/README.md +0 -16
  385. package/evals/fixtures/tiny-saas-app/docs/auth-review.md +0 -15
  386. package/evals/fixtures/tiny-saas-app/docs/billing-policy.md +0 -10
  387. package/evals/fixtures/tiny-saas-app/docs/copy-conventions.md +0 -15
  388. package/evals/fixtures/tiny-saas-app/docs/specs/settings-title.md +0 -13
  389. package/evals/fixtures/tiny-saas-app/src/auth-middleware.ts +0 -12
  390. package/evals/fixtures/tiny-saas-app/src/billing.ts +0 -20
  391. package/evals/fixtures/tiny-saas-app/src/login-copy.ts +0 -1
  392. package/evals/fixtures/tiny-saas-app/src/settings-page.ts +0 -1
  393. package/evals/fixtures/tiny-saas-app/tests/auth-middleware.test.md +0 -8
  394. package/evals/fixtures/tiny-saas-app/tests/billing.test.md +0 -8
  395. package/evals/fixtures/tiny-saas-app/tests/copy.test.md +0 -7
  396. package/evals/fixtures/tiny-setup-agents-app/AGENTS.md +0 -6
  397. package/evals/fixtures/tiny-setup-both-app/AGENTS.md +0 -5
  398. package/evals/fixtures/tiny-setup-both-app/CLAUDE.md +0 -5
  399. package/evals/fixtures/tiny-setup-claude-app/CLAUDE.md +0 -6
  400. package/evals/fixtures/tiny-setup-codex-rules-app/.codex/rules/shell.rules +0 -5
  401. package/evals/fixtures/tiny-setup-codex-rules-app/AGENTS.md +0 -6
  402. package/evals/fixtures/tiny-setup-conflict-app/AGENTS.md +0 -6
  403. package/evals/fixtures/tiny-setup-existing-block-app/AGENTS.md +0 -14
  404. package/evals/fixtures/tiny-setup-four-rules-app/CLAUDE.md +0 -6
  405. package/evals/fixtures/tiny-skill-dev-app/README.md +0 -10
  406. package/evals/fixtures/tiny-skill-dev-app/evals/prompts/review-pr-001.txt +0 -8
  407. package/evals/fixtures/tiny-skill-dev-app/evals/reports/review-pr-failure.md +0 -17
  408. package/evals/fixtures/tiny-skill-dev-app/skills/review-pr/SKILL.md +0 -13
  409. package/evals/fixtures/tiny-team-app/README.md +0 -5
  410. package/evals/fixtures/tiny-team-app/docs/product.md +0 -6
  411. package/evals/fixtures/tiny-team-app/src/teams.ts +0 -9
  412. package/evals/fixtures/tiny-team-app/src/users.ts +0 -5
  413. package/evals/fixtures/tiny-team-app/tests/teams.test.md +0 -7
  414. package/evals/fixtures/tiny-verify-app/README.md +0 -5
  415. package/evals/fixtures/tiny-verify-app/src/settings-nav.ts +0 -1
  416. package/evals/fixtures/tiny-verify-app/src/settings-page.ts +0 -1
  417. package/evals/fixtures/tiny-verify-app/tests/settings.test.md +0 -7
  418. package/evals/prompts/aon-001.txt +0 -2
  419. package/evals/prompts/byp-001.txt +0 -2
  420. package/evals/prompts/byp-002.txt +0 -2
  421. package/evals/prompts/cap-001.txt +0 -3
  422. package/evals/prompts/cap-002.txt +0 -3
  423. package/evals/prompts/cmd-001.txt +0 -3
  424. package/evals/prompts/cmd-002.txt +0 -3
  425. package/evals/prompts/cmd-003.txt +0 -6
  426. package/evals/prompts/cmd-004.txt +0 -5
  427. package/evals/prompts/cmd-005.txt +0 -5
  428. package/evals/prompts/cmd-006.txt +0 -5
  429. package/evals/prompts/cmd-007.txt +0 -5
  430. package/evals/prompts/cmd-008.txt +0 -5
  431. package/evals/prompts/cmd-009.txt +0 -8
  432. package/evals/prompts/cmd-010.txt +0 -5
  433. package/evals/prompts/cmd-011.txt +0 -5
  434. package/evals/prompts/cmd-012.txt +0 -5
  435. package/evals/prompts/cmd-013.txt +0 -5
  436. package/evals/prompts/cmd-014.txt +0 -7
  437. package/evals/prompts/cmd-015.txt +0 -7
  438. package/evals/prompts/cmd-016.txt +0 -5
  439. package/evals/prompts/cmt-001.txt +0 -1
  440. package/evals/prompts/cmt-002.txt +0 -1
  441. package/evals/prompts/cmt-003.txt +0 -1
  442. package/evals/prompts/cmt-004.txt +0 -1
  443. package/evals/prompts/cmt-005.txt +0 -1
  444. package/evals/prompts/dfd-001.txt +0 -3
  445. package/evals/prompts/dia-001.txt +0 -3
  446. package/evals/prompts/dia-002.txt +0 -3
  447. package/evals/prompts/dia-003.txt +0 -3
  448. package/evals/prompts/dis-001.txt +0 -5
  449. package/evals/prompts/dis-002.txt +0 -5
  450. package/evals/prompts/esk-001.txt +0 -5
  451. package/evals/prompts/esk-002.txt +0 -5
  452. package/evals/prompts/esk-003.txt +0 -5
  453. package/evals/prompts/grc-001.txt +0 -3
  454. package/evals/prompts/hof-001.txt +0 -3
  455. package/evals/prompts/hof-002.txt +0 -1
  456. package/evals/prompts/hof-003.txt +0 -1
  457. package/evals/prompts/hof-004.txt +0 -1
  458. package/evals/prompts/hof-005.txt +0 -1
  459. package/evals/prompts/ivg-001.txt +0 -1
  460. package/evals/prompts/ivg-002.txt +0 -11
  461. package/evals/prompts/ivg-003.txt +0 -1
  462. package/evals/prompts/ivg-004.txt +0 -1
  463. package/evals/prompts/mode-001.txt +0 -3
  464. package/evals/prompts/mode-002.txt +0 -1
  465. package/evals/prompts/mode-003.txt +0 -3
  466. package/evals/prompts/mode-004.txt +0 -3
  467. package/evals/prompts/mode-005.txt +0 -3
  468. package/evals/prompts/mode-006.txt +0 -3
  469. package/evals/prompts/mode-007.txt +0 -3
  470. package/evals/prompts/otr-001.txt +0 -3
  471. package/evals/prompts/otr-002.txt +0 -3
  472. package/evals/prompts/rar-001.txt +0 -5
  473. package/evals/prompts/rar-002.txt +0 -3
  474. package/evals/prompts/rar-003.txt +0 -6
  475. package/evals/prompts/rar-004.txt +0 -15
  476. package/evals/prompts/rar-005.txt +0 -10
  477. package/evals/prompts/rbr-001.txt +0 -3
  478. package/evals/prompts/rbr-002.txt +0 -3
  479. package/evals/prompts/rbr-003.txt +0 -7
  480. package/evals/prompts/rev-001.txt +0 -3
  481. package/evals/prompts/rev-002.txt +0 -5
  482. package/evals/prompts/rev-003.txt +0 -6
  483. package/evals/prompts/rev-004.txt +0 -1
  484. package/evals/prompts/rev-005.txt +0 -15
  485. package/evals/prompts/rev-006.txt +0 -5
  486. package/evals/prompts/stp-001.txt +0 -1
  487. package/evals/prompts/stp-002.txt +0 -1
  488. package/evals/prompts/stp-003.txt +0 -1
  489. package/evals/prompts/stp-004.txt +0 -1
  490. package/evals/prompts/stp-005.txt +0 -1
  491. package/evals/prompts/stp-006.txt +0 -1
  492. package/evals/prompts/stp-007.txt +0 -1
  493. package/evals/prompts/stp-008.txt +0 -1
  494. package/evals/prompts/stp-009.txt +0 -1
  495. package/evals/prompts/stp-010.txt +0 -1
  496. package/evals/prompts/stp-011.txt +0 -1
  497. package/evals/prompts/vfy-003.txt +0 -1
  498. package/evals/prompts/wpl-001.txt +0 -3
  499. package/evals/prompts/wpl-002.txt +0 -5
  500. package/evals/prompts/wpl-003.txt +0 -3
  501. package/evals/prompts/wpl-004.txt +0 -5
  502. package/evals/prompts/wpl-005.txt +0 -1
  503. package/evals/prompts/wrc-001.txt +0 -10
  504. package/evals/prompts/wrc-002.txt +0 -3
  505. package/evals/prompts/wsk-001.txt +0 -6
  506. package/evals/prompts/wsp-001.txt +0 -5
  507. package/evals/prompts/wsp-002.txt +0 -15
  508. package/evals/prompts/wsp-003.txt +0 -3
  509. package/evals/prompts/wsp-004.txt +0 -15
  510. package/evals/prompts/wsp-005.txt +0 -5
  511. package/evals/prompts/wsp-006.txt +0 -12
  512. package/evals/prompts/wsp-007.txt +0 -9
  513. package/evals/prompts/wsp-008.txt +0 -13
  514. package/evals/prompts/xpl-001.txt +0 -1
  515. package/evals/prompts/xpl-002.txt +0 -3
  516. package/evals/prompts/xpl-003.txt +0 -3
  517. package/evals/prompts/xpl-004.txt +0 -3
  518. package/evals/prompts/xpl-005.txt +0 -5
  519. package/evals/prompts/xpl-006.txt +0 -3
  520. package/evals/registries/adversarial-evals.json +0 -39
  521. package/evals/registries/fixture-evals.json +0 -2513
  522. package/evals/registries/handoff-evals.json +0 -74
  523. package/evals/registries/legacy-evals.json +0 -147
  524. package/evals/registries/skill-evidence.json +0 -193
  525. package/evals/registries/verify-work-evals.json +0 -46
  526. package/evals/reports/acceptance/v0.1-acceptance-report.md +0 -130
  527. package/evals/reports/by-command-surface/command-surface-1-report.md +0 -45
  528. package/evals/reports/by-command-surface/command-surface-10-report.md +0 -76
  529. package/evals/reports/by-command-surface/command-surface-11-report.md +0 -77
  530. package/evals/reports/by-command-surface/command-surface-12-report.md +0 -91
  531. package/evals/reports/by-command-surface/command-surface-13-report.md +0 -64
  532. package/evals/reports/by-command-surface/command-surface-14-report.md +0 -66
  533. package/evals/reports/by-command-surface/command-surface-15-report.md +0 -124
  534. package/evals/reports/by-command-surface/command-surface-16-report.md +0 -39
  535. package/evals/reports/by-command-surface/command-surface-2-report.md +0 -80
  536. package/evals/reports/by-command-surface/command-surface-3-report.md +0 -86
  537. package/evals/reports/by-command-surface/command-surface-4-report.md +0 -76
  538. package/evals/reports/by-command-surface/command-surface-5-report.md +0 -80
  539. package/evals/reports/by-command-surface/command-surface-6-report.md +0 -74
  540. package/evals/reports/by-command-surface/command-surface-7-report.md +0 -84
  541. package/evals/reports/by-command-surface/command-surface-8-report.md +0 -79
  542. package/evals/reports/by-command-surface/command-surface-9-report.md +0 -79
  543. package/evals/reports/by-command-surface/command-surface-matrix.md +0 -91
  544. package/evals/reports/by-skill/bypass-1-report.md +0 -124
  545. package/evals/reports/by-skill/capture-decisions-1-report.md +0 -99
  546. package/evals/reports/by-skill/capture-decisions-2-report.md +0 -102
  547. package/evals/reports/by-skill/commit-work-1-report.md +0 -165
  548. package/evals/reports/by-skill/commit-work-2-report.md +0 -109
  549. package/evals/reports/by-skill/design-for-depth-1-report.md +0 -95
  550. package/evals/reports/by-skill/diagnose-failure-1-report.md +0 -91
  551. package/evals/reports/by-skill/diagnose-failure-2-report.md +0 -94
  552. package/evals/reports/by-skill/discover-1-report.md +0 -118
  553. package/evals/reports/by-skill/evaluate-skill-2-report.md +0 -71
  554. package/evals/reports/by-skill/execute-plan-1-report.md +0 -103
  555. package/evals/reports/by-skill/execute-plan-2-report.md +0 -112
  556. package/evals/reports/by-skill/execute-plan-3-report.md +0 -87
  557. package/evals/reports/by-skill/grill-context-1-report.md +0 -103
  558. package/evals/reports/by-skill/handoff-1-report.md +0 -123
  559. package/evals/reports/by-skill/handoff-2-report.md +0 -105
  560. package/evals/reports/by-skill/handoff-3-report.md +0 -100
  561. package/evals/reports/by-skill/handoff-7-report.md +0 -77
  562. package/evals/reports/by-skill/interview-gate-1-report.md +0 -63
  563. package/evals/reports/by-skill/interview-gate-2-report.md +0 -105
  564. package/evals/reports/by-skill/mode-contract-1-report.md +0 -74
  565. package/evals/reports/by-skill/mode-contract-2-report.md +0 -87
  566. package/evals/reports/by-skill/mode-contract-3-report.md +0 -65
  567. package/evals/reports/by-skill/mode-contract-4-report.md +0 -71
  568. package/evals/reports/by-skill/mode-contract-5-report.md +0 -70
  569. package/evals/reports/by-skill/output-router-1-report.md +0 -76
  570. package/evals/reports/by-skill/output-router-2-report.md +0 -65
  571. package/evals/reports/by-skill/research-1-report.md +0 -144
  572. package/evals/reports/by-skill/research-brief-1-report.md +0 -94
  573. package/evals/reports/by-skill/research-brief-2-report.md +0 -75
  574. package/evals/reports/by-skill/review-artifact-1-report.md +0 -91
  575. package/evals/reports/by-skill/review-artifact-2-report.md +0 -79
  576. package/evals/reports/by-skill/review-artifact-3-report.md +0 -120
  577. package/evals/reports/by-skill/review-artifact-4-report.md +0 -89
  578. package/evals/reports/by-skill/review-artifact-5-report.md +0 -66
  579. package/evals/reports/by-skill/review-work-1-report.md +0 -85
  580. package/evals/reports/by-skill/review-work-2-report.md +0 -87
  581. package/evals/reports/by-skill/review-work-3-report.md +0 -104
  582. package/evals/reports/by-skill/review-work-4-report.md +0 -108
  583. package/evals/reports/by-skill/review-work-5-report.md +0 -73
  584. package/evals/reports/by-skill/review-work-6-report.md +0 -66
  585. package/evals/reports/by-skill/setup-freeflow-1-report.md +0 -199
  586. package/evals/reports/by-skill/setup-freeflow-2-report.md +0 -105
  587. package/evals/reports/by-skill/setup-freeflow-3-report.md +0 -105
  588. package/evals/reports/by-skill/setup-freeflow-4-report.md +0 -88
  589. package/evals/reports/by-skill/setup-freeflow-5-report.md +0 -115
  590. package/evals/reports/by-skill/skill-authoring-1-report.md +0 -81
  591. package/evals/reports/by-skill/verify-work-1-report.md +0 -111
  592. package/evals/reports/by-skill/verify-work-2-report.md +0 -120
  593. package/evals/reports/by-skill/workflow-1-report.md +0 -65
  594. package/evals/reports/by-skill/write-plan-1-report.md +0 -123
  595. package/evals/reports/by-skill/write-plan-2-report.md +0 -98
  596. package/evals/reports/by-skill/write-plan-3-report.md +0 -110
  597. package/evals/reports/by-skill/write-spec-1-report.md +0 -88
  598. package/evals/reports/by-skill/write-spec-2-report.md +0 -89
  599. package/evals/reports/by-skill/write-spec-3-report.md +0 -90
  600. package/evals/reports/by-skill/write-spec-4-report.md +0 -167
  601. package/evals/reports/harness/claude-harness-1-report.md +0 -103
  602. package/evals/reports/harness/eval-harness-1-report.md +0 -91
  603. package/evals/reports/iterations/iteration-1-report.md +0 -71
  604. package/evals/reports/iterations/iteration-2-report.md +0 -78
  605. package/evals/reports/iterations/iteration-3-report.md +0 -118
  606. package/evals/reports/iterations/iteration-4-report.md +0 -131
  607. package/evals/reports/iterations/iteration-5-report.md +0 -149
  608. package/evals/reports/iterations/iteration-6-report.md +0 -136
  609. package/evals/reports/runtime/always-on-runtime-1-report.md +0 -107
  610. package/evals/reports/runtime/context-mode-normalized-benchmark-1-report.md +0 -52
  611. package/evals/reports/runtime/context-mode-real-deep-baseline-1-report.md +0 -410
  612. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-report.md +0 -203
  613. package/evals/reports/runtime/context-mode-real-deep-final-slice-11-review.md +0 -51
  614. package/evals/reports/runtime/context-saving-native-tools-baseline-1-report.md +0 -99
  615. package/evals/reports/runtime/eryx-python-compatibility-reprobe-1-report.md +0 -166
  616. package/evals/reports/runtime/eryx-python-proof-spike-1-report.md +0 -96
  617. package/evals/reports/runtime/eryx-python-proof-spike-2-report.md +0 -151
  618. package/evals/reports/runtime/eryx-python-script-derive-execution-1-report.md +0 -233
  619. package/evals/reports/runtime/jq-script-derive-execution-1-report.md +0 -215
  620. package/evals/reports/runtime/jq-wasm-proof-spike-1-report.md +0 -67
  621. package/evals/reports/runtime/output-router-benchmark-1-report.md +0 -81
  622. package/evals/reports/runtime/output-router-capture-derive-provider-eval-1-report.md +0 -35
  623. package/evals/reports/runtime/output-router-codex-large-index-sqlite-adhoc-report.md +0 -150
  624. package/evals/reports/runtime/output-router-codex-qa-benchmark-1-report.md +0 -46
  625. package/evals/reports/runtime/output-router-command-benchmark-1-report.md +0 -62
  626. package/evals/reports/runtime/output-router-index-benchmark-1-report.md +0 -64
  627. package/evals/reports/runtime/output-router-regression-1-report.md +0 -81
  628. package/evals/reports/runtime/output-router-transform-eval-1-report.md +0 -28
  629. package/evals/reports/runtime/pi-observed-routing-eval-1-report.md +0 -44
  630. package/evals/reports/runtime/quickjs-script-derive-execution-1-report.md +0 -159
  631. package/evals/reports/runtime/quickjs-wasi-proof-spike-1-report.md +0 -55
  632. package/evals/reports/runtime/render-format-row-protocol-1-report.md +0 -58
  633. package/evals/reports/runtime/script-sandbox-adapter-selection-review-1-report.md +0 -84
  634. package/evals/reports/runtime/script-sandbox-probe-resource-hardening-1-report.md +0 -51
  635. package/evals/reports/runtime/storage-policy-benchmark-1-report.md +0 -176
  636. package/evals/reports/runtime/vault-index-storage-spike-1-report.md +0 -39
  637. package/evals/reports/runtime/workflow-context-hook-1-report.md +0 -82
  638. package/evals/runbooks/adversarial-eval-runbook.md +0 -28
  639. package/evals/runbooks/fixture-eval-runbook.md +0 -137
  640. package/evals/runbooks/handoff-eval-runbook.md +0 -22
  641. package/evals/runbooks/verify-work-eval-runbook.md +0 -27
  642. package/evals/scripts/audit-command-surface.sh +0 -160
  643. package/evals/scripts/check-activation-contract.sh +0 -121
  644. package/evals/scripts/check-runtime-context-hook.sh +0 -183
  645. package/evals/scripts/grade-fixture-eval.sh +0 -446
  646. package/evals/scripts/run-claude-fixture-eval.sh +0 -26
  647. package/evals/scripts/run-codex-fixture-eval.sh +0 -26
  648. package/evals/scripts/run-eryx-python-proof-spike.js +0 -559
  649. package/evals/scripts/run-fixture-eval-by-id.sh +0 -23
  650. package/evals/scripts/run-fixture-eval.sh +0 -464
  651. package/evals/scripts/run-jq-wasm-proof-spike.js +0 -299
  652. package/evals/scripts/run-output-router-transform-eval.js +0 -115
  653. package/evals/scripts/run-pi-observed-routing-eval.js +0 -478
  654. package/evals/scripts/run-quickjs-wasi-proof-spike.js +0 -339
  655. package/evals/scripts/skill-evidence.sh +0 -316
  656. package/evals/scripts/validate-release-metadata.sh +0 -486
  657. package/evals/suites/v0.1-acceptance-suite.md +0 -131
  658. package/pi-extension/dist/host-producer-identification.js +0 -186
  659. package/pi-extension/dist/mcp-config.js +0 -30
  660. package/pi-extension/dist/native-safety-net.js +0 -209
  661. package/pi-extension/dist/observed-tool-routing.js +0 -114
  662. package/pi-extension/dist/renderers.js +0 -835
  663. package/pi-extension/dist/router-tools.js +0 -438
  664. package/pi-extension/dist/runtime-context.js +0 -288
  665. package/pi-extension/dist/schemas.js +0 -254
  666. package/pi-extension/dist/status.js +0 -449
  667. package/pi-extension/dist/utils.js +0 -467
  668. package/pi-extension/src/runtime-context.ts +0 -347
  669. package/pi-extension/src/status.ts +0 -505
  670. package/plugin-docs/README.md +0 -40
  671. package/plugin-docs/adr/0001-three-modes.md +0 -27
  672. package/plugin-docs/adr/0002-evals-before-hooks.md +0 -17
  673. package/plugin-docs/adr/0003-release-boundary.md +0 -21
  674. package/plugin-docs/adr/0004-discover-replaces-shallow-discovery-skills.md +0 -26
  675. package/plugin-docs/adr/README.md +0 -10
  676. package/plugin-docs/architecture.md +0 -89
  677. package/plugin-docs/output-router.md +0 -396
  678. package/plugin-docs/release-evidence.md +0 -77
  679. package/plugin-docs/skills.md +0 -44
  680. package/plugin-docs/workflow.md +0 -57
  681. package/skills/diagnose-failure/references/feedback-loop-catalog.md +0 -25
  682. package/skills/discover/ARTIFACT-DESTINATIONS.md +0 -58
  683. package/skills/discover/CHECKPOINTS.md +0 -74
  684. package/skills/discover/SKILL.md +0 -173
  685. package/skills/evaluate-skill/references/eval-patterns.md +0 -41
  686. package/skills/evaluate-skill/references/grading-priority.md +0 -33
  687. package/skills/execute-plan/SKILL.md +0 -133
  688. package/skills/execute-plan/references/execution-map.md +0 -122
  689. package/skills/interview-gate/SKILL.md +0 -90
  690. package/skills/setup-freeflow/references/output-router-setup.md +0 -102
  691. package/skills/workflow/references/workflow-map.md +0 -104
  692. /package/pi-extension/src/{mcp-config.ts → integrations/mcp-config.ts} +0 -0
@@ -1,151 +0,0 @@
1
- # Eryx Python Sandbox Proof Spike 2 Report
2
-
3
- > **Date:** 2026-06-24
4
- > **Status:** Passed temp-patched proof spike for Python candidate
5
- > **Scope:** Proof-only evaluation. No Freeflow script execution path is enabled.
6
-
7
- ## Candidate
8
-
9
- - Package: `@bsull/eryx@0.5.0`
10
- - License: MIT OR Apache-2.0
11
- - Package root: `/tmp/freeflow-eryx-proof-run-87KaRW/node_modules/@bsull/eryx`
12
- - Preview2 shim: `@bytecodealliance/preview2-shim@0.17.9`
13
- - Preview2 shim root: `/tmp/freeflow-eryx-proof-run-87KaRW/node_modules/@bytecodealliance/preview2-shim`
14
- - Patched temp copy: `/var/folders/2x/tsrlzqfx3ld_fn5bmr1_3l600000gn/T/freeflow-eryx-proof-0QeDOW`
15
- - WASM shards: 10
16
- - WASM total bytes: 44664377
17
- - Timeout: 500ms via Worker termination
18
- - Output cap: 4096 bytes across stdout + stderr before Worker result crosses to parent
19
- - Worker resource limits: `maxOldGenerationSizeMb=256`, `maxYoungGenerationSizeMb=32`
20
- - Node requirement: parent process started with `--experimental-wasm-jspi`
21
-
22
- ## Package Files
23
-
24
- - `eryx-sandbox.core.wasm` — 14217612 bytes
25
- - `eryx-sandbox.core10.wasm` — 5014 bytes
26
- - `eryx-sandbox.core2.wasm` — 66749 bytes
27
- - `eryx-sandbox.core3.wasm` — 1949270 bytes
28
- - `eryx-sandbox.core4.wasm` — 5780 bytes
29
- - `eryx-sandbox.core5.wasm` — 8408 bytes
30
- - `eryx-sandbox.core6.wasm` — 27258519 bytes
31
- - `eryx-sandbox.core7.wasm` — 1090852 bytes
32
- - `eryx-sandbox.core8.wasm` — 10145 bytes
33
- - `eryx-sandbox.core9.wasm` — 52028 bytes
34
- - `eryx-sandbox.d.ts` — 4086 bytes
35
- - `eryx-sandbox.js` — 1069741 bytes
36
- - `index.d.ts` — 4463 bytes
37
- - `index.js` — 7898 bytes
38
- - `interfaces/eryx-net-tcp.d.ts` — 1056 bytes
39
- - `interfaces/eryx-net-tls.d.ts` — 931 bytes
40
- - `interfaces/wasi-cli-environment.d.ts` — 209 bytes
41
- - `interfaces/wasi-cli-exit.d.ts` — 177 bytes
42
- - `interfaces/wasi-cli-stderr.d.ts` — 163 bytes
43
- - `interfaces/wasi-cli-stdin.d.ts` — 158 bytes
44
- - `interfaces/wasi-cli-stdout.d.ts` — 163 bytes
45
- - `interfaces/wasi-cli-terminal-input.d.ts` — 176 bytes
46
- - `interfaces/wasi-cli-terminal-output.d.ts` — 178 bytes
47
- - `interfaces/wasi-cli-terminal-stderr.d.ts` — 207 bytes
48
- - `interfaces/wasi-cli-terminal-stdin.d.ts` — 201 bytes
49
- - `interfaces/wasi-cli-terminal-stdout.d.ts` — 207 bytes
50
- - `interfaces/wasi-clocks-monotonic-clock.d.ts` — 374 bytes
51
- - `interfaces/wasi-clocks-wall-clock.d.ts` — 200 bytes
52
- - `interfaces/wasi-filesystem-preopens.d.ts` — 194 bytes
53
- - `interfaces/wasi-filesystem-types.d.ts` — 5405 bytes
54
- - `interfaces/wasi-io-error.d.ts` — 185 bytes
55
- - `interfaces/wasi-io-poll.d.ts` — 254 bytes
56
- - `interfaces/wasi-io-streams.d.ts` — 1170 bytes
57
- - `interfaces/wasi-random-insecure-seed.d.ts` — 108 bytes
58
- - `interfaces/wasi-random-insecure.d.ts` — 166 bytes
59
- - `interfaces/wasi-random-random.d.ts` — 148 bytes
60
- - `interfaces/wasi-sockets-instance-network.d.ts` — 173 bytes
61
- - `interfaces/wasi-sockets-ip-name-lookup.d.ts` — 617 bytes
62
- - `interfaces/wasi-sockets-network.d.ts` — 2190 bytes
63
- - `interfaces/wasi-sockets-tcp-create-socket.d.ts` — 360 bytes
64
- - `interfaces/wasi-sockets-tcp.d.ts` — 1954 bytes
65
- - `interfaces/wasi-sockets-udp-create-socket.d.ts` — 360 bytes
66
- - `interfaces/wasi-sockets-udp.d.ts` — 1698 bytes
67
- - `package.json` — 1135 bytes
68
- - `python-stdlib.tar.gz` — 3082487 bytes
69
- - `README.md` — 4505 bytes
70
- - `shims/callbacks.d.ts` — 1376 bytes
71
- - `shims/callbacks.js` — 3991 bytes
72
- - `shims/net.js` — 1074 bytes
73
- - `shims/sockets.js` — 1068 bytes
74
- - `stdlib-loader.js` — 5523 bytes
75
-
76
- ## WASM Shard Hashes
77
-
78
- - `eryx-sandbox.core.wasm` — 14217612 bytes — `63083db02cba1b628c5a939458c8e8c374f99b89d113a2c0670f715404944939`
79
- - `eryx-sandbox.core10.wasm` — 5014 bytes — `66a00b7099832c6cc85d42439e8bed880cdb0912ddb7f82e8b73c54fec89522e`
80
- - `eryx-sandbox.core2.wasm` — 66749 bytes — `d632b1884e0e376335a101e98d92c2d9279eda8dc8e196d8cf5114036d9512be`
81
- - `eryx-sandbox.core3.wasm` — 1949270 bytes — `f963fd45a753db5c8b33f8247b3f77aa045aca4ad516c580a6f70930def8d579`
82
- - `eryx-sandbox.core4.wasm` — 5780 bytes — `d67ce024ccf6d4191760a543888eccc9e1bca7ac4bfd68a794a693d238efd3f4`
83
- - `eryx-sandbox.core5.wasm` — 8408 bytes — `aa9012bd695b1f9d1bf3a770a547c2a3705be47cf00d4b45898b019c1f937799`
84
- - `eryx-sandbox.core6.wasm` — 27258519 bytes — `fde75b7a2c2d57630d1966cc0aa64f52fd6e48c17f10236e5c09da11e28ba8cc`
85
- - `eryx-sandbox.core7.wasm` — 1090852 bytes — `ca9450bf30ecf9243966168e043c72dc1c69208ef8ca5a0d4993b57bbc000b37`
86
- - `eryx-sandbox.core8.wasm` — 10145 bytes — `c7d356a9a1e71d26a37be99c9ee536d40484e36522b6447a625c6270e95637b0`
87
- - `eryx-sandbox.core9.wasm` — 52028 bytes — `e13ac26656e8c6fb5862961821a115df02009f1b9bd8b16acda4c1826b5f484d`
88
-
89
- ## Import Patch
90
-
91
- The runner copies `@bsull/eryx` and `@bytecodealliance/preview2-shim` into a temp `node_modules` tree, then rewrites Eryx's generated preview2-shim imports from bare package subpaths to explicit browser/in-memory shim files:
92
-
93
- - `@bytecodealliance/preview2-shim/cli` → `../../@bytecodealliance/preview2-shim/lib/browser/cli.js`
94
- - `@bytecodealliance/preview2-shim/clocks` → `../../@bytecodealliance/preview2-shim/lib/browser/clocks.js`
95
- - `@bytecodealliance/preview2-shim/filesystem` → `../../@bytecodealliance/preview2-shim/lib/browser/filesystem.js`
96
- - `@bytecodealliance/preview2-shim/io` → `../../@bytecodealliance/preview2-shim/lib/browser/io.js`
97
- - `@bytecodealliance/preview2-shim/random` → `../../@bytecodealliance/preview2-shim/lib/browser/random.js`
98
-
99
- This avoids Node resolving `@bytecodealliance/preview2-shim/filesystem` to the host-filesystem-oriented Node shim that lacks `_setFileData`.
100
-
101
- The runner also replaces the temp-copied Eryx `shims/net.js` with an equivalent deny-only shim that records TCP/TLS attempts and returns `not-permitted`. This makes the network proof check the wrapper denial event instead of accepting any generic socket failure.
102
-
103
- ## Import Probe
104
-
105
- - Status: success
106
- - Exports: `Sandbox`, `_fileTree`, `execute`, `setCallbackHandler`, `setCallbacks`, `setOutputHandler`, `setResultVariable`, `setTraceHandler`
107
- - Error: none
108
-
109
- ## Positive API Probe
110
-
111
- - Status: success
112
- - Output: `INFO setup\nERROR target`
113
- - Error: none
114
-
115
- ## Required Proof Results
116
-
117
- | Proof | Result | Evidence |
118
- | --- | --- | --- |
119
- | env_access_denied | pass | ambient host environment and secret sentinel were not exposed; status=success; exitCode=0; durationMs=82; outputBytes=89; stdoutBytes=89; stderrBytes=0; rawStdoutBytes=89; rawStderrBytes=0; truncated=false; outputFiles=0; networkEvents=none; error=none |
120
- | home_access_denied | pass | home directory was unavailable and home/secret file contents were not exposed; status=error; exitCode=1; durationMs=128; outputBytes=790; stdoutBytes=460; stderrBytes=330; rawStdoutBytes=460; rawStderrBytes=330; truncated=false; outputFiles=0; networkEvents=none; error=Traceback (most recent call last):\n File "<string>", line 1, in <module>\n File "<string>", line 179, in _eryx_exec\n File "<user>", line 2, in <module>\n File "/python-stdlib/pathlib/__init__.py", line 1266, in home\n raise RuntimeError("Could not determine home directory.")\nRuntimeError: Could not determine home directory.\n |
121
- | repo_access_denied | pass | repo file contents were not exposed; status=error; exitCode=1; durationMs=126; outputBytes=1146; stdoutBytes=483; stderrBytes=663; rawStdoutBytes=483; rawStderrBytes=663; truncated=false; outputFiles=0; networkEvents=none; error=Traceback (most recent call last):\n File "<string>", line 1, in <module>\n File "<string>", line 179, in _eryx_exec\n File "<user>", line 2, in <module>\n File "/python-stdlib/pathlib/__init__.py", line 792, in read_text\n with self.open(mode='r', encoding=encoding, errors=errors, newline=newline) as f:\n ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/pathlib/__init__.py", line 776, in open\n return io.open(self, mode, buffering, encoding, errors, newline)\n ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nNotADirectoryError: [Errno 54] Not a directory: '/workspace/package.json'\n |
122
- | vault_access_denied | pass | vault records were not exposed; status=success; exitCode=0; durationMs=117; outputBytes=462; stdoutBytes=462; stderrBytes=0; rawStdoutBytes=462; rawStderrBytes=0; truncated=false; outputFiles=0; networkEvents=none; error=none |
123
- | network_access_denied | pass | outbound network attempt reached the instrumented Eryx shim and was denied as not-permitted; status=error; exitCode=1; durationMs=219; outputBytes=4096; stdoutBytes=2048; stderrBytes=2048; rawStdoutBytes=3795; rawStderrBytes=2442; truncated=true; outputFiles=0; networkEvents=tcp.connect:not-permitted,tls.upgrade:not-permitted,tls.write:not-permitted; error=Traceback (most recent call last):\n File "/python-stdlib/urllib/request.py", line 1321, in do_open\n h.request(req.get_method(), req.selector, req.data, headers,\n ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n encode_chunked=req.has_header('Transfer-encoding'))\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/http/client.py", line 1338, in request\n self._send_request(method, url, body, headers, encode_chunked)\n ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/http/client.py", line 1384, in _send_request\n self.endheaders(body, encode_chunked=encode_chunked)\n ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/http/client.py", line 1333, in endheaders\n self._send_output(message_body, encode_chunked=encode_chunked)\n ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/http/client.py", line 1093, in _send_output\n self.send(msg)\n ~~~~~~~~~^^^^^\n File "/python-stdlib/http/client.py", line 1057, in send\n self.sock.sendall(data)\n ~~~~~~~~~~~~~~~~~^^^^^^\n File "<ssl_eryx>", line 220, in sendall\n File "<socket_eryx>", line 359, in sendall\nsocket.error: Connection closed\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "<string>", line 1, in <module>\n File "<string>", line 179, in _eryx_exec\n File "<user>", line 2, in <module>\n File "/python-stdlib/urllib/request.py", line 187, in urlopen\n return opener.open(url, data, timeout)\n ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/urllib/request.py", line 487, in open\n response = self._open(req, data)\n File "/python-stdlib/urllib/request.py", line 504, in _open\n result = self._call_chain(self.handle_open, protocol, protocol +\n '_open', req)\n File "/python-stdlib/urllib/request.py", line 464, in _call_chain\n result = func(*args)\n File "/python-stdlib/urllib/request.py", line 1369, in https_open\n return self.do_open(http.client.HTTPSConnection, req,\n ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n context=self._context)\n ^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/urllib/request.py", line 1324, in do_open\n raise URLError(err)\nurllib.error.URLError: <urlopen error Connection closed>\n |
124
- | input_read_only | pass | input mutation was not possible through filesystem paths; status=error; exitCode=1; durationMs=120; outputBytes=1143; stdoutBytes=483; stderrBytes=660; rawStdoutBytes=483; rawStderrBytes=660; truncated=false; outputFiles=0; networkEvents=none; error=Traceback (most recent call last):\n File "<string>", line 1, in <module>\n File "<string>", line 179, in _eryx_exec\n File "<user>", line 2, in <module>\n File "/python-stdlib/pathlib/__init__.py", line 814, in write_text\n with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:\n ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/python-stdlib/pathlib/__init__.py", line 776, in open\n return io.open(self, mode, buffering, encoding, errors, newline)\n ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nNotADirectoryError: [Errno 54] Not a directory: '/input/test_log.txt'\n |
125
- | output_escape_denied | pass | output symlink attempt stayed inside ignored in-memory filesystem and no output files were collected; status=success; exitCode=0; durationMs=119; outputBytes=454; stdoutBytes=454; stderrBytes=0; rawStdoutBytes=454; rawStderrBytes=0; truncated=false; outputFiles=0; networkEvents=none; error=none |
126
- | stdout_stderr_bounded | pass | stdout/stderr flood was capped before crossing from Worker to parent; status=success; exitCode=0; durationMs=103; outputBytes=4096; stdoutBytes=2048; stderrBytes=2048; rawStdoutBytes=100999; rawStderrBytes=100999; truncated=true; outputFiles=0; networkEvents=none; error=none |
127
- | timeout_enforced | pass | infinite loop was stopped by Worker termination; status=timed_out; exitCode=null; durationMs=502; outputBytes=0; stdoutBytes=0; stderrBytes=0; rawStdoutBytes=0; rawStderrBytes=0; truncated=false; outputFiles=0; networkEvents=none; error=none |
128
-
129
- ## Notes
130
-
131
- - This proof runner uses a temporary installed `@bsull/eryx` package root passed explicitly by the caller.
132
- - It does not add repo dependencies and does not wire Python into `freeflow_derive` execution.
133
- - It performs a temp-copy import rewrite and deny-only network-shim replacement. A product adapter would need an explicit package-root wrapper and focused security review before enabling this path.
134
- - Timeout proof uses Node Worker termination because Eryx's high-level JS API does not expose timeout or fuel controls.
135
- - Output proof caps what crosses from Worker to parent; Eryx/Python can still materialize large strings inside the Worker before the wrapper truncates them.
136
- - The runner intentionally collects no output files, matching the current QuickJS/jq product adapters. In-memory filesystem writes and symlink attempts are ignored unless their contents reach stdout/stderr.
137
- - The runner returns bounded error messages rather than JS host stack traces so proof output does not leak repo/home paths through adapter diagnostics.
138
- - The runner overrides Worker `console` before importing Eryx so preview2 browser shim debug logs are captured and bounded instead of inherited by host stdout.
139
- - Passing this spike supports candidate feasibility only. Product execution still requires adapter implementation, probe caching, review, and runtime-status wiring.
140
-
141
- ## Required Proof Set
142
-
143
- - env_access_denied
144
- - home_access_denied
145
- - repo_access_denied
146
- - vault_access_denied
147
- - network_access_denied
148
- - input_read_only
149
- - output_escape_denied
150
- - stdout_stderr_bounded
151
- - timeout_enforced
@@ -1,233 +0,0 @@
1
- # Eryx Python Script Derive Execution Report 1
2
-
3
- > **Date:** 2026-06-24
4
- > **Status:** Passed Eryx Python execution smoke
5
- > **Scope:** Slice 17 execution engine. Python can execute only through an explicitly registered/provided Eryx adapter; script derive remains disabled by default.
6
-
7
- ## Summary
8
-
9
- - Added dependency-free Eryx Python adapter support in router core.
10
- - Adapter loading is explicit. The Pi adapter discovers Python only when `FREEFLOW_ERYX_ROOT` points to an installed `@bsull/eryx` package root.
11
- - Eryx requires the parent Node process to run with `--experimental-wasm-jspi`.
12
- - No `package.json` dependency was added.
13
- - No runtime artifacts are installed, downloaded, or vendored during execution.
14
- - Script derive remains disabled by default.
15
-
16
- ## Adapter Surface
17
-
18
- The Eryx adapter runs CPython/WASM inside a Node Worker. The guest Python program receives:
19
-
20
- - `read_text(alias)` — reads copied vault-source input by alias,
21
- - `write_text(text)` — writes to captured stdout,
22
- - `sources` — a Python dict keyed by source alias.
23
-
24
- The adapter does not expose host repo paths, home paths, vault paths, ambient environment variables, process APIs, package loading, direct host filesystem access, or network APIs.
25
-
26
- The adapter uses a temp-copied Eryx package tree and rewrites Eryx's preview2-shim imports to browser/in-memory preview2 shims. It also replaces the temp-copied Eryx network shim with a deny-only shim that records TCP/TLS `not-permitted` events during proofs.
27
-
28
- Timeouts are enforced by Worker termination. Output is capped before the Worker result crosses to the host. If output is truncated, execution returns a structured `derive_execution_failure`; truncated stdout is not persisted as an exact result.
29
-
30
- Residual caveat: Eryx/Python can still generate large strings inside the Worker before wrapper truncation. The Worker uses resource limits and timeout termination, but output cap enforcement is not streaming inside Python itself.
31
-
32
- ## Product Path
33
-
34
- Execution requires all of:
35
-
36
- - `scriptDerive.enabled=true`,
37
- - `scriptDerive.languages` includes `python`,
38
- - a proof-backed Eryx adapter is registered/provided,
39
- - Pi can discover `FREEFLOW_ERYX_ROOT`, or the caller passes the adapter directly,
40
- - the parent Node process has `--experimental-wasm-jspi`,
41
- - `network="off"`,
42
- - raw script persistence remains disabled.
43
-
44
- ## Actual Adapter Proof Selection
45
-
46
- With package root:
47
-
48
- ```text
49
- /tmp/freeflow-eryx-proof-run-87KaRW/node_modules/@bsull/eryx
50
- ```
51
-
52
- `probeScriptSandboxAdapters` reported:
53
-
54
- ```json
55
- {
56
- "probeStatus": "available",
57
- "availableLanguages": ["python"],
58
- "passed": 9,
59
- "failed": 0
60
- }
61
- ```
62
-
63
- Command evidence: `ffout_b223c00fa8c1f7b82ce6ab15`.
64
-
65
- The dedicated proof report is `evals/reports/runtime/eryx-python-proof-spike-2-report.md`.
66
-
67
- ## Product Execution Smoke
68
-
69
- A real `freeflowDerive` smoke used a vaulted stdout source:
70
-
71
- ```text
72
- alpha
73
- ```
74
-
75
- and Python code:
76
-
77
- ```python
78
- write_text(read_text('log').upper())
79
- ```
80
-
81
- Result excerpt:
82
-
83
- ```json
84
- {
85
- "deriveStatus": "ok",
86
- "failure": null,
87
- "outputId": "ffout_569831f03c6f0d79149a2c58",
88
- "raw": "ALPHA",
89
- "leakedScript": false
90
- }
91
- ```
92
-
93
- Raw script text was not persisted in the routed result; the operation records only `codeSha256`.
94
-
95
- Command evidence: `ffout_b223c00fa8c1f7b82ce6ab15`.
96
-
97
- ## Output Cap Regression Smoke
98
-
99
- A real Eryx product-path smoke used a configured adapter proof cap of `4096` bytes and a per-call `limits.maxOutputBytes=10` with:
100
-
101
- ```python
102
- write_text('x' * 100)
103
- ```
104
-
105
- Result excerpt:
106
-
107
- ```json
108
- {
109
- "failure": "derive_execution_failure",
110
- "message": "Script derive python failed. Eryx Python execution exceeded maxOutputBytes. stdoutBytes=5 stderrBytes=0.",
111
- "outputId": null,
112
- "recoverability": "none"
113
- }
114
- ```
115
-
116
- This confirms truncated Python output is not persisted as a successful exact result. The adapter splits the output cap between stdout and stderr so both streams retain bounded evidence under flood conditions.
117
-
118
- Command evidence: `ffout_ef09471811902881360f0416`.
119
-
120
- ## Timeout Smoke
121
-
122
- A real Eryx timeout smoke used:
123
-
124
- ```python
125
- while True:
126
- pass
127
- ```
128
-
129
- with per-call `limits.timeoutMs=250`.
130
-
131
- Result excerpt:
132
-
133
- ```json
134
- {
135
- "failure": "derive_execution_failure",
136
- "message": "Script derive python timed_out. Eryx Python execution exceeded timeoutMs. stdoutBytes=0 stderrBytes=0.",
137
- "outputId": null,
138
- "recoverability": "none"
139
- }
140
- ```
141
-
142
- Command evidence: `ffout_ef09471811902881360f0416`.
143
-
144
- ## Targeted Tests
145
-
146
- ```text
147
- npm run build && node --test router/tests/derive.test.js router/tests/script-sandbox.test.js router/tests/pi-extension-derive.test.js router/tests/pi-extension.test.js
148
- ```
149
-
150
- Result:
151
-
152
- ```text
153
- 69 passed, 1 skipped
154
- ```
155
-
156
- The skipped test is the optional Eryx integration path, which requires `FREEFLOW_ERYX_ROOT` and Node `--experimental-wasm-jspi`.
157
-
158
- Coverage included:
159
-
160
- - script derive remains disabled by default through the Pi public tool,
161
- - invalid `FREEFLOW_ERYX_ROOT` fails closed as adapter unavailable,
162
- - Pi status and derive wiring still build and pass focused tests,
163
- - proof fixture coverage remains complete.
164
-
165
- ## Optional Eryx Integration Test
166
-
167
- ```text
168
- FREEFLOW_ERYX_ROOT=/tmp/freeflow-eryx-proof-run-87KaRW/node_modules/@bsull/eryx \
169
- node --experimental-wasm-jspi --test router/tests/derive.test.js --test-name-pattern "Eryx adapter"
170
- ```
171
-
172
- Result:
173
-
174
- ```text
175
- 25 tests passed, including:
176
- ✔ freeflowDerive script operation executes Python through discovered Eryx adapter when configured
177
- ```
178
-
179
- Command evidence: `ffout_b1fbb65e96d814268c0aa0c2`; post-full-tree-fingerprint rerun evidence: `ffout_f7ccfed70b4323788ce921c7`; post-mutation-check rerun evidence: `ffout_4a0268f37ecc9436804fcab5`.
180
-
181
- ## Pi Status Smoke
182
-
183
- With `FREEFLOW_ERYX_ROOT=/tmp/freeflow-eryx-proof-run-87KaRW/node_modules/@bsull/eryx` and Node `--experimental-wasm-jspi`, `buildFreeflowStatusReport` reports:
184
-
185
- ```json
186
- {
187
- "enabled": false,
188
- "executionStatus": "disabled",
189
- "adapterAvailable": true,
190
- "adapterStatus": "available",
191
- "availableLanguages": ["python"],
192
- "registeredAdapters": [
193
- { "id": "eryx-python", "version": "0.5.0", "languages": ["python"] }
194
- ]
195
- }
196
- ```
197
-
198
- This proves explicit adapter discovery does not enable script execution while `scriptDerive.enabled` remains false.
199
-
200
- Command evidence: `ffout_d321f5f2d24d548d935e5a9f`.
201
-
202
- ## Full Router Suite
203
-
204
- ```text
205
- env -u FREEFLOW_ERYX_ROOT npm run test:router
206
- ```
207
-
208
- Result:
209
-
210
- ```text
211
- 278 passed, 1 skipped
212
- ```
213
-
214
- The skipped test is the optional Eryx integration path.
215
-
216
- Command evidence: `ffout_8b08d9c1893023bb3a74c702`; post-full-tree-fingerprint rerun evidence: `ffout_002c6ed4ff25a713538bb738`; post-mutation-check rerun evidence: `ffout_d88f87f47353dfa6351e22a9`.
217
-
218
- ## Current Availability
219
-
220
- | Language | Status |
221
- | --- | --- |
222
- | JavaScript | Product execution path implemented for explicitly registered/provided QuickJS adapter roots; disabled by default. |
223
- | Python | Product execution path implemented for explicitly registered/provided Eryx adapter roots plus Node JSPI; disabled by default. |
224
- | jq | Product execution path implemented for explicitly registered/provided jq-wasm adapter roots; disabled by default. |
225
-
226
- ## Remaining Gates
227
-
228
- Before presenting script derive as generally available:
229
-
230
- 1. Keep setup UX explicit: Freeflow must not install or download adapter packages.
231
- 2. Run full router suite and focused review for this implementation slice.
232
- 3. Document Eryx's JSPI and in-Worker large-output caveats in user-facing docs.
233
- 4. Keep network, raw-script persistence, direct repo inputs, and output-file collection disabled unless separate proof/review approves them.
@@ -1,215 +0,0 @@
1
- # jq Script Derive Execution Report 1
2
-
3
- > **Date:** 2026-06-24
4
- > **Status:** Passed jq execution smoke
5
- > **Scope:** Partial Slice 17 execution engine. jq can execute only through an explicitly registered/provided jq-wasm adapter; script derive remains disabled by default.
6
-
7
- ## Summary
8
-
9
- - Added dependency-free jq-wasm adapter support in router core.
10
- - Adapter loading is explicit. The Pi adapter discovers jq only when `FREEFLOW_JQ_WASM_ROOT` points to an installed `jq-wasm` package root.
11
- - No `package.json` dependency was added.
12
- - No runtime artifacts are installed, downloaded, or vendored during execution.
13
- - Python remains unavailable.
14
-
15
- ## Adapter Surface
16
-
17
- The jq adapter runs `jq-wasm` inside a Node Worker. The guest jq program receives a JSON object keyed by source alias. For a source alias `log`, the jq program reads `.log`.
18
-
19
- The adapter does not expose host repo paths, home paths, vault paths, ambient process APIs, package loading to guest jq, or network APIs.
20
-
21
- Timeouts are enforced by Worker termination. Output is capped before the Worker result crosses to the host. If output is truncated, execution returns a structured `derive_execution_failure`; truncated stdout is not persisted as an exact result.
22
-
23
- Residual caveat: `jq-wasm` can still generate large strings inside the Worker before wrapper truncation. The Worker uses resource limits and timeout termination, but output cap enforcement is not streaming inside jq itself.
24
-
25
- ## Product Path
26
-
27
- Execution requires all of:
28
-
29
- - `scriptDerive.enabled=true`,
30
- - `scriptDerive.languages` includes `jq`,
31
- - a proof-backed jq adapter is registered/provided,
32
- - Pi can discover `FREEFLOW_JQ_WASM_ROOT`, or the caller passes the adapter directly,
33
- - `network="off"`,
34
- - raw script persistence remains disabled.
35
-
36
- ## Actual Adapter Proof Selection
37
-
38
- With package root:
39
-
40
- ```text
41
- /tmp/freeflow-sandbox-candidates-xVAJBl/node_modules/jq-wasm
42
- ```
43
-
44
- `probeScriptSandboxAdapters` reported:
45
-
46
- ```json
47
- {
48
- "adapterAvailable": true,
49
- "availableLanguages": ["jq"],
50
- "failedProofs": [],
51
- "passedProofs": [
52
- "env_access_denied",
53
- "home_access_denied",
54
- "repo_access_denied",
55
- "vault_access_denied",
56
- "network_access_denied",
57
- "input_read_only",
58
- "output_escape_denied",
59
- "stdout_stderr_bounded",
60
- "timeout_enforced"
61
- ],
62
- "reason": "jq-wasm passed every required jq sandbox proof.",
63
- "runtime": {
64
- "name": "jq-wasm",
65
- "version": "1.2.0-jq-1.8.2 entry:4a5bd373cb59"
66
- }
67
- }
68
- ```
69
-
70
- ## Product Execution Smoke
71
-
72
- A real `freeflowDerive` smoke used a vaulted stdout source:
73
-
74
- ```text
75
- INFO alpha
76
- ERROR beta
77
- ```
78
-
79
- and jq code:
80
-
81
- ```jq
82
- .log | split("\n") | map(select(test("ERROR"))) | .[]
83
- ```
84
-
85
- Result excerpt:
86
-
87
- ```json
88
- {
89
- "toolStatus": "ok",
90
- "operation": {
91
- "kind": "script",
92
- "language": "jq",
93
- "codeSha256": "sha256_e5e69ac26a62710aa0313ddaa13e6619c21bd0c1e2adffa0230b63de82c68c0c"
94
- },
95
- "summary": "Script derive jq completed with 12 stdout bytes and 0 stderr bytes.",
96
- "derived": "\"ERROR beta\"",
97
- "persistence": {
98
- "status": "vaulted",
99
- "recoverability": "exact"
100
- }
101
- }
102
- ```
103
-
104
- Raw script text was not persisted in the routed result; the operation records only `codeSha256`.
105
-
106
- ## Output Cap Regression Smoke
107
-
108
- A real jq product-path smoke used a configured adapter proof cap of `4096` bytes and a per-call `limits.maxOutputBytes=10` with:
109
-
110
- ```jq
111
- range(0; 1000) | "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
112
- ```
113
-
114
- Result excerpt:
115
-
116
- ```json
117
- {
118
- "toolStatus": "ok",
119
- "failure": {
120
- "kind": "derive_execution_failure",
121
- "message": "Script derive jq failed. jq-wasm execution exceeded maxOutputBytes. stdoutBytes=10 stderrBytes=0."
122
- },
123
- "deriveExecution": {
124
- "status": "failed",
125
- "failureKind": "derive_execution_failure"
126
- },
127
- "persistence": {
128
- "status": "not_persisted",
129
- "recoverability": "none"
130
- }
131
- }
132
- ```
133
-
134
- This confirms truncated jq output is not persisted as a successful exact result.
135
-
136
- ## Timeout Smoke
137
-
138
- A real jq timeout smoke used:
139
-
140
- ```jq
141
- def loop: loop; loop
142
- ```
143
-
144
- Result excerpt:
145
-
146
- ```json
147
- {
148
- "toolStatus": "ok",
149
- "failure": {
150
- "kind": "derive_execution_failure",
151
- "message": "Script derive jq timed_out. jq-wasm execution exceeded timeoutMs. stdoutBytes=0 stderrBytes=0."
152
- },
153
- "deriveExecution": {
154
- "status": "failed",
155
- "failureKind": "derive_execution_failure"
156
- },
157
- "persistence": {
158
- "status": "not_persisted",
159
- "recoverability": "none"
160
- }
161
- }
162
- ```
163
-
164
- ## Pi Status Smoke
165
-
166
- With `FREEFLOW_JQ_WASM_ROOT=/tmp/freeflow-sandbox-candidates-xVAJBl/node_modules/jq-wasm`, `buildFreeflowStatusReport` reports:
167
-
168
- ```json
169
- {
170
- "enabled": false,
171
- "executionStatus": "disabled",
172
- "adapterAvailable": true,
173
- "availableLanguages": ["jq"],
174
- "registeredAdapters": [
175
- { "id": "jq-wasm", "version": "1.2.0-jq-1.8.2", "languages": ["jq"] }
176
- ]
177
- }
178
- ```
179
-
180
- This proves explicit adapter discovery does not enable script execution while `scriptDerive.enabled` remains false.
181
-
182
- ## Tests
183
-
184
- Targeted check passed:
185
-
186
- ```text
187
- npm run build:router && node --test router/tests/derive.test.js router/tests/script-sandbox.test.js
188
- ```
189
-
190
- Result: 32/32 targeted tests passed.
191
-
192
- Coverage includes:
193
-
194
- - script derive remains disabled by default,
195
- - jq execution works through a registered proof-backed adapter,
196
- - adapter timeout/failure returns structured `derive_execution_failure`,
197
- - output-limit failures return structured `derive_execution_failure` without exact recovery,
198
- - invalid `FREEFLOW_JQ_WASM_ROOT` fails closed as adapter unavailable.
199
-
200
- ## Current Availability
201
-
202
- | Language | Status |
203
- | --- | --- |
204
- | JavaScript | Product execution path implemented for explicitly registered/provided QuickJS adapter roots; disabled by default. |
205
- | jq | Product execution path implemented for explicitly registered/provided jq-wasm adapter roots; disabled by default. |
206
- | Python | Unavailable. Eryx candidate blocked before proofs. |
207
-
208
- ## Remaining Work
209
-
210
- Before script derive can be described as complete across target languages:
211
-
212
- 1. Run focused implementation/security review for jq product execution.
213
- 2. Run the full router suite.
214
- 3. Find or fix a Python WASM candidate.
215
- 4. Keep public docs precise: JavaScript and jq are explicit opt-in; Python remains unavailable; script derive is disabled by default.
@@ -1,67 +0,0 @@
1
- # jq-wasm Sandbox Proof Spike Report
2
-
3
- > **Date:** 2026-06-24
4
- > **Status:** Passed proof spike for jq candidate
5
- > **Scope:** Proof-only evaluation. No Freeflow script execution path is enabled.
6
-
7
- ## Candidate
8
-
9
- - Package: `jq-wasm@1.2.0-jq-1.8.2`
10
- - License: MIT
11
- - Package root: `/tmp/freeflow-sandbox-candidates-xVAJBl/node_modules/jq-wasm`
12
- - Entry point: `/tmp/freeflow-sandbox-candidates-xVAJBl/node_modules/jq-wasm/dist/index.js`
13
- - Runtime version: `jq-1.8.2-dirty`
14
- - Timeout: 500ms via Worker termination
15
- - Output cap: 4096 bytes across stdout + stderr before Worker result crosses to host
16
- - Worker resource limits: `maxOldGenerationSizeMb=64`, `maxYoungGenerationSizeMb=16`
17
-
18
- ## Package Files
19
-
20
- - `dist/build/jq.d.ts` — 118 bytes
21
- - `dist/build/jq.js` — 1181780 bytes
22
- - `dist/index.d.ts` — 1133 bytes
23
- - `dist/index.js` — 3242 bytes
24
- - `LICENSE` — 1051 bytes
25
- - `package.json` — 1981 bytes
26
- - `README.md` — 3088 bytes
27
-
28
- ## Positive API Probe
29
-
30
- - Status: success
31
- - Output: `"INFO setup\nERROR target\n"`
32
- - Error: none
33
-
34
- ## Required Proof Results
35
-
36
- | Proof | Result | Evidence |
37
- | --- | --- | --- |
38
- | env_access_denied | pass | jq exposed only fixed runtime env, not ambient host env or secrets; status=success; exitCode=0; durationMs=29; outputBytes=126; stdoutBytes=126; stderrBytes=0; rawStdoutBytes=126; rawStderrBytes=0; truncated=false |
39
- | home_access_denied | pass | home/secret paths were not exposed; status=success; exitCode=3; durationMs=27; outputBytes=61; stdoutBytes=0; stderrBytes=61; rawStdoutBytes=0; rawStderrBytes=61; truncated=false |
40
- | repo_access_denied | pass | repo files were not exposed; status=success; exitCode=3; durationMs=29; outputBytes=61; stdoutBytes=0; stderrBytes=61; rawStdoutBytes=0; rawStderrBytes=61; truncated=false |
41
- | vault_access_denied | pass | vault records were not exposed; status=success; exitCode=3; durationMs=27; outputBytes=62; stdoutBytes=0; stderrBytes=62; rawStdoutBytes=0; rawStderrBytes=62; truncated=false |
42
- | network_access_denied | pass | jq has no ambient network primitive; synthetic fetch errored boundedly; status=success; exitCode=5; durationMs=30; outputBytes=48; stdoutBytes=0; stderrBytes=48; rawStdoutBytes=0; rawStderrBytes=48; truncated=false |
43
- | input_read_only | pass | input mutation include was unavailable; status=success; exitCode=3; durationMs=29; outputBytes=71; stdoutBytes=0; stderrBytes=71; rawStdoutBytes=0; rawStderrBytes=71; truncated=false |
44
- | output_escape_denied | pass | output escape include was unavailable; status=success; exitCode=3; durationMs=28; outputBytes=63; stdoutBytes=0; stderrBytes=63; rawStdoutBytes=0; rawStderrBytes=63; truncated=false |
45
- | stdout_stderr_bounded | pass | stdout and stderr flood output was truncated before crossing the worker boundary; status=success; exitCode=0; durationMs=353; outputBytes=4096; stdoutBytes=2048; stderrBytes=2048; rawStdoutBytes=5299999; rawStderrBytes=6399999; truncated=true |
46
- | timeout_enforced | pass | recursive jq loop was interrupted by worker termination; status=timed_out; exitCode=null; durationMs=501; outputBytes=0; stdoutBytes=0; stderrBytes=0; rawStdoutBytes=0; rawStderrBytes=0; truncated=false |
47
-
48
- ## Notes
49
-
50
- - This proof runner uses a temporary installed `jq-wasm` package root passed explicitly by the caller.
51
- - It does not add repo dependencies and does not wire the adapter into `freeflow_derive` execution.
52
- - Timeout proof uses Node Worker termination because in-thread recursive jq blocks the event loop.
53
- - Output proof caps what crosses the Worker boundary; `jq-wasm` itself can still generate large in-Worker strings before the wrapper truncates them.
54
- - Passing this spike only supports jq adapter feasibility; Python remains unavailable until a Python candidate passes proofs.
55
- - Before product execution, this must still go through implementation/security review and source-plan update.
56
-
57
- ## Required Proof Set
58
-
59
- - env_access_denied
60
- - home_access_denied
61
- - repo_access_denied
62
- - vault_access_denied
63
- - network_access_denied
64
- - input_read_only
65
- - output_escape_denied
66
- - stdout_stderr_bounded
67
- - timeout_enforced