@hassangameryt/freeflow 0.3.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +4 -4
- package/.claude-plugin/plugin.json +6 -2
- package/.codex-plugin/plugin.json +9 -9
- package/README.md +190 -127
- package/assets/feedback-loop.png +0 -0
- package/assets/freeflow_workflow_map.png +0 -0
- package/assets/workflow.png +0 -0
- package/{skills → capabilities}/output-router/SKILL.md +21 -17
- package/{skills → capabilities}/output-router/references/safety-policy.md +1 -1
- package/capabilities/output-router/references/setup.md +110 -0
- package/command-surface.json +62 -7
- package/hooks/freeflow-runtime-context.mjs +572 -154
- package/hooks/hooks.json +26 -1
- package/hooks/tests/check-runtime-context-hook.sh +375 -0
- package/package.json +24 -24
- package/pi-extension/dist/index.js +264 -80
- package/pi-extension/dist/integrations/mcp-config.js +29 -0
- package/pi-extension/dist/output-router/host-producer-identification.js +232 -0
- package/pi-extension/dist/output-router/native-safety-net.js +216 -0
- package/pi-extension/dist/output-router/observed-tool-routing.js +122 -0
- package/pi-extension/dist/output-router/renderers.js +1023 -0
- package/pi-extension/dist/output-router/router-tools.js +554 -0
- package/pi-extension/dist/output-router/schemas.js +383 -0
- package/pi-extension/dist/output-router/status.js +744 -0
- package/pi-extension/dist/output-router/utils.js +502 -0
- package/pi-extension/dist/runtime/runtime-context.js +798 -0
- package/pi-extension/dist/settings/settings-tui.js +314 -0
- package/pi-extension/dist/settings/settings-ui.js +1487 -0
- package/pi-extension/freeflow/index.js +1 -0
- package/pi-extension/src/index.ts +230 -56
- package/pi-extension/src/{host-producer-identification.ts → output-router/host-producer-identification.ts} +37 -7
- package/pi-extension/src/{native-safety-net.ts → output-router/native-safety-net.ts} +8 -12
- package/pi-extension/src/{observed-tool-routing.ts → output-router/observed-tool-routing.ts} +19 -11
- package/pi-extension/src/{renderers.ts → output-router/renderers.ts} +246 -55
- package/pi-extension/src/{router-tools.ts → output-router/router-tools.ts} +130 -29
- package/pi-extension/src/{schemas.ts → output-router/schemas.ts} +171 -43
- package/pi-extension/src/output-router/status.ts +791 -0
- package/pi-extension/src/{utils.ts → output-router/utils.ts} +80 -18
- package/pi-extension/src/runtime/runtime-context.ts +895 -0
- package/pi-extension/src/settings/settings-tui.ts +408 -0
- package/pi-extension/src/settings/settings-ui.ts +1730 -0
- package/pi-extension/tests/integration/pi-extension.test.js +4020 -0
- package/pi-extension/tests/output-router/observed-producers.test.js +132 -0
- package/pi-extension/tests/output-router/search.test.js +722 -0
- package/pi-extension/tests/settings/settings-tui.test.js +243 -0
- package/router/dist/benchmarks/benchmark-harness.d.ts +16 -13
- package/router/dist/benchmarks/benchmark-harness.js +62 -62
- package/router/dist/benchmarks/benchmarks.d.ts +80 -73
- package/router/dist/benchmarks/benchmarks.js +860 -761
- package/router/dist/benchmarks/codex-qa-benchmarks.d.ts +71 -67
- package/router/dist/benchmarks/codex-qa-benchmarks.js +429 -390
- package/router/dist/benchmarks/command-benchmarks.d.ts +107 -92
- package/router/dist/benchmarks/command-benchmarks.js +616 -571
- package/router/dist/benchmarks/context-mode-normalized-benchmarks.d.ts +64 -56
- package/router/dist/benchmarks/context-mode-normalized-benchmarks.js +679 -613
- package/router/dist/benchmarks/context-mode-real-deep-benchmark.d.ts +103 -95
- package/router/dist/benchmarks/context-mode-real-deep-benchmark.js +1188 -1103
- package/router/dist/benchmarks/context-saving-baseline.d.ts +47 -41
- package/router/dist/benchmarks/context-saving-baseline.js +405 -374
- package/router/dist/benchmarks/index-benchmarks.d.ts +74 -69
- package/router/dist/benchmarks/index-benchmarks.js +761 -661
- package/router/dist/benchmarks/storage-policy-benchmarks.d.ts +80 -69
- package/router/dist/benchmarks/storage-policy-benchmarks.js +549 -469
- package/router/dist/benchmarks/vault-index-benchmarks.d.ts +43 -34
- package/router/dist/benchmarks/vault-index-benchmarks.js +294 -229
- package/router/dist/config/config.d.ts +62 -55
- package/router/dist/config/config.js +552 -373
- package/router/dist/config/router-contract.d.ts +2 -2
- package/router/dist/config/router-contract.js +38 -38
- package/router/dist/config/schema.d.ts +19 -10
- package/router/dist/config/schema.js +586 -538
- package/router/dist/config/types.d.ts +413 -361
- package/router/dist/config/types.js +19 -8
- package/router/dist/evidence/bounded-evidence.d.ts +26 -26
- package/router/dist/evidence/bounded-evidence.js +265 -252
- package/router/dist/evidence/evidence-range-selector.d.ts +16 -14
- package/router/dist/evidence/evidence-range-selector.js +192 -191
- package/router/dist/evidence/evidence-search.d.ts +18 -16
- package/router/dist/evidence/evidence-search.js +401 -377
- package/router/dist/evidence/evidence.d.ts +17 -17
- package/router/dist/evidence/evidence.js +163 -159
- package/router/dist/evidence/failure-contracts.d.ts +25 -15
- package/router/dist/evidence/failure-contracts.js +132 -126
- package/router/dist/evidence/line-ranges.d.ts +17 -15
- package/router/dist/evidence/line-ranges.js +18 -18
- package/router/dist/experiments/local-index.d.ts +47 -41
- package/router/dist/experiments/local-index.js +477 -458
- package/router/dist/local/local-traversal.d.ts +18 -0
- package/router/dist/local/local-traversal.js +334 -0
- package/router/dist/processing/engine.d.ts +94 -70
- package/router/dist/processing/engine.js +462 -364
- package/router/dist/processing/reducers.d.ts +208 -194
- package/router/dist/processing/reducers.js +1434 -1256
- package/router/dist/processing/renderers.d.ts +24 -18
- package/router/dist/processing/renderers.js +161 -152
- package/router/dist/processing/scripts.d.ts +80 -67
- package/router/dist/processing/scripts.js +343 -290
- package/router/dist/repo/repo-traversal.d.ts +9 -9
- package/router/dist/repo/repo-traversal.js +205 -202
- package/router/dist/routing/observed-reducers.d.ts +25 -19
- package/router/dist/routing/observed-reducers.js +559 -545
- package/router/dist/routing/observed-routing.d.ts +23 -14
- package/router/dist/routing/observed-routing.js +319 -318
- package/router/dist/routing/parsers.d.ts +10 -10
- package/router/dist/routing/parsers.js +454 -406
- package/router/dist/routing/run-filters.d.ts +45 -41
- package/router/dist/routing/run-filters.js +215 -207
- package/router/dist/routing/run-reducers.d.ts +45 -26
- package/router/dist/routing/run-reducers.js +102 -90
- package/router/dist/sandbox/adapter-roots.d.ts +34 -26
- package/router/dist/sandbox/adapter-roots.js +43 -41
- package/router/dist/sandbox/eryx-python-adapter.d.ts +9 -5
- package/router/dist/sandbox/eryx-python-adapter.js +659 -588
- package/router/dist/sandbox/jq-wasm-adapter.d.ts +3 -3
- package/router/dist/sandbox/jq-wasm-adapter.js +319 -311
- package/router/dist/sandbox/quickjs-wasi-adapter.d.ts +9 -5
- package/router/dist/sandbox/quickjs-wasi-adapter.js +336 -333
- package/router/dist/sandbox/script-sandbox.d.ts +105 -84
- package/router/dist/sandbox/script-sandbox.js +294 -260
- package/router/dist/setup/script-transform-adapters.d.ts +23 -19
- package/router/dist/setup/script-transform-adapters.js +223 -204
- package/router/dist/tools/batch.d.ts +27 -16
- package/router/dist/tools/batch.js +493 -416
- package/router/dist/tools/run.d.ts +47 -35
- package/router/dist/tools/run.js +1172 -1087
- package/router/dist/tools/search.d.ts +53 -31
- package/router/dist/tools/search.js +1224 -1065
- package/router/dist/transform/engine.d.ts +121 -96
- package/router/dist/transform/engine.js +1853 -1736
- package/router/dist/vault/vault-index.d.ts +97 -78
- package/router/dist/vault/vault-index.js +347 -342
- package/router/dist/vault/vault.d.ts +109 -74
- package/router/dist/vault/vault.js +528 -533
- package/runtime/interaction-contract.md +16 -0
- package/skills/bypass/SKILL.md +52 -38
- package/skills/commit-work/SKILL.md +54 -46
- package/skills/commit-work/references/staging-decisions.md +46 -24
- package/skills/decision-gate/SKILL.md +48 -0
- package/skills/design-for-depth/SKILL.md +75 -59
- package/skills/design-for-depth/references/design-pressure-signals.md +56 -7
- package/skills/design-for-depth/references/interface-design-loop.md +156 -0
- package/skills/design-for-depth/references/software-design-philosophy.md +32 -3
- package/skills/diagnose-failure/SKILL.md +65 -60
- package/skills/diagnose-failure/references/diagnostic-loop-catalog.md +36 -0
- package/skills/diagnose-failure/references/flaky-and-performance.md +30 -18
- package/skills/discuss/SKILL.md +121 -0
- package/skills/discuss/references/checkpoints.md +66 -0
- package/skills/evaluate-skill/SKILL.md +62 -53
- package/skills/evaluate-skill/references/definition-schema.md +345 -0
- package/skills/evaluate-skill/references/evaluation-design.md +86 -0
- package/skills/evaluate-skill/references/execution-and-evidence.md +85 -0
- package/skills/evaluate-skill/references/review-and-revision.md +51 -0
- package/skills/evaluate-skill/scripts/lib/definitions.mjs +296 -0
- package/skills/evaluate-skill/scripts/lib/evidence.mjs +46 -0
- package/skills/evaluate-skill/scripts/lib/grade-comparison.mjs +60 -0
- package/skills/evaluate-skill/scripts/lib/grade-interaction.mjs +148 -0
- package/skills/evaluate-skill/scripts/lib/grade-support.mjs +126 -0
- package/skills/evaluate-skill/scripts/lib/grade-workspace.mjs +192 -0
- package/skills/evaluate-skill/scripts/lib/grade.mjs +56 -0
- package/skills/evaluate-skill/scripts/lib/pi.mjs +489 -0
- package/skills/evaluate-skill/scripts/lib/process.mjs +79 -0
- package/skills/evaluate-skill/scripts/lib/rpc.mjs +444 -0
- package/skills/evaluate-skill/scripts/lib/runner.mjs +346 -0
- package/skills/evaluate-skill/scripts/lib/sandbox.mjs +409 -0
- package/skills/evaluate-skill/scripts/lib/view.mjs +337 -0
- package/skills/evaluate-skill/scripts/pi-guard.mjs +164 -0
- package/skills/evaluate-skill/scripts/skill-eval.mjs +101 -0
- package/skills/execute-work/SKILL.md +99 -0
- package/skills/execute-work/references/code-practices.md +78 -0
- package/skills/execute-work/references/execution-loop.md +77 -0
- package/skills/finish-branch/SKILL.md +117 -0
- package/skills/finish-branch/references/integration-options.md +80 -0
- package/skills/handoff/SKILL.md +65 -39
- package/skills/handoff/references/templates.md +104 -27
- package/skills/launch-work/SKILL.md +116 -0
- package/skills/launch-work/references/launch-readiness.md +85 -0
- package/skills/migration-work/SKILL.md +115 -0
- package/skills/migration-work/references/migration-lifecycle.md +82 -0
- package/skills/mode-contract/SKILL.md +72 -52
- package/skills/release-work/SKILL.md +118 -0
- package/skills/release-work/references/release-evidence.md +78 -0
- package/skills/review-artifact/SKILL.md +77 -69
- package/skills/review-artifact/references/reviewer-prompt.md +133 -42
- package/skills/review-work/SKILL.md +82 -57
- package/skills/review-work/references/reviewer-prompt.md +97 -60
- package/skills/review-work/references/security-risk-lens.md +67 -0
- package/skills/setup-freeflow/SKILL.md +54 -68
- package/skills/setup-freeflow/references/activation-contract.md +84 -69
- package/skills/setup-freeflow/references/host-setup.md +50 -17
- package/skills/simplify-code/SKILL.md +80 -0
- package/skills/simplify-code/references/simplification-patterns.md +70 -0
- package/skills/tdd/SKILL.md +96 -0
- package/skills/tdd/references/test-design.md +96 -0
- package/skills/track-work/SKILL.md +84 -0
- package/skills/track-work/references/working-record-schema.md +378 -0
- package/skills/verify-work/SKILL.md +59 -38
- package/skills/verify-work/references/browser-runtime-evidence.md +55 -0
- package/skills/verify-work/references/integration-evidence.md +51 -0
- package/skills/verify-work/references/performance-evidence.md +57 -0
- package/skills/workflow/SKILL.md +53 -59
- package/skills/workflow/references/domain-skill-composition.md +57 -0
- package/skills/workflow/references/workflow-loop.md +92 -0
- package/skills/write-plan/SKILL.md +85 -58
- package/skills/write-plan/references/plan-shapes.md +132 -46
- package/skills/write-skill/SKILL.md +82 -51
- package/skills/write-skill/references/activation-boundaries.md +62 -0
- package/skills/write-skill/references/agent-first-instructions.md +87 -0
- package/skills/write-skill/references/progressive-disclosure.md +63 -0
- package/skills/write-skill/scripts/lib/skill-author.mjs +653 -0
- package/skills/write-skill/scripts/skill-author.mjs +104 -0
- package/skills/write-spec/SKILL.md +65 -54
- package/skills/write-spec/references/artifact-standards.md +60 -41
- package/skills/write-spec/references/decision-records.md +77 -0
- package/skills/write-spec/references/spec-shapes.md +119 -60
- package/evals/README.md +0 -32
- package/evals/fixtures/copy-conventions.md +0 -31
- package/evals/fixtures/output-router/README.md +0 -12
- package/evals/fixtures/output-router/failed-command-output.txt +0 -9
- package/evals/fixtures/output-router/large-router-manual.md +0 -63
- package/evals/fixtures/output-router/native-large-output.txt +0 -40
- package/evals/fixtures/output-router/noisy-test-output.txt +0 -30
- package/evals/fixtures/output-router/verification-output.txt +0 -8
- package/evals/fixtures/tiny-artifact-review-app/README.md +0 -5
- package/evals/fixtures/tiny-artifact-review-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-artifact-review-app/docs/specs/billing-webhook-api-draft.md +0 -21
- package/evals/fixtures/tiny-artifact-review-app/docs/specs/failed-payment-grace-period.md +0 -24
- package/evals/fixtures/tiny-artifact-review-app/docs/specs/immediate-downgrade.md +0 -17
- package/evals/fixtures/tiny-artifact-review-app/tests/billing.test.md +0 -8
- package/evals/fixtures/tiny-commit-ambiguous-app/.freeflow-eval-setup.sh +0 -15
- package/evals/fixtures/tiny-commit-ambiguous-app/README.md +0 -3
- package/evals/fixtures/tiny-commit-ambiguous-app/src/billing.ts +0 -5
- package/evals/fixtures/tiny-commit-ambiguous-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-commit-ambiguous-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-commit-app/.freeflow-eval-setup.sh +0 -14
- package/evals/fixtures/tiny-commit-app/README.md +0 -3
- package/evals/fixtures/tiny-commit-app/docs/specs/settings-title.md +0 -5
- package/evals/fixtures/tiny-commit-app/src/billing.ts +0 -5
- package/evals/fixtures/tiny-commit-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-commit-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-commit-everything-app/.freeflow-eval-setup.sh +0 -15
- package/evals/fixtures/tiny-commit-everything-app/README.md +0 -3
- package/evals/fixtures/tiny-commit-everything-app/docs/specs/settings-title.md +0 -5
- package/evals/fixtures/tiny-commit-everything-app/src/billing.ts +0 -5
- package/evals/fixtures/tiny-commit-everything-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-commit-everything-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-commit-staged-app/.freeflow-eval-setup.sh +0 -15
- package/evals/fixtures/tiny-commit-staged-app/README.md +0 -3
- package/evals/fixtures/tiny-commit-staged-app/docs/specs/settings-title.md +0 -5
- package/evals/fixtures/tiny-commit-staged-app/src/billing.ts +0 -5
- package/evals/fixtures/tiny-commit-staged-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-commit-staged-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-commit-staged-mixed-app/.freeflow-eval-setup.sh +0 -15
- package/evals/fixtures/tiny-commit-staged-mixed-app/README.md +0 -3
- package/evals/fixtures/tiny-commit-staged-mixed-app/docs/billing-policy.md +0 -3
- package/evals/fixtures/tiny-commit-staged-mixed-app/docs/release-notes.md +0 -3
- package/evals/fixtures/tiny-commit-staged-mixed-app/src/billing.ts +0 -5
- package/evals/fixtures/tiny-dashboard-stale-app/README.md +0 -11
- package/evals/fixtures/tiny-dashboard-stale-app/src/api-client.ts +0 -13
- package/evals/fixtures/tiny-dashboard-stale-app/src/dashboard-data.ts +0 -19
- package/evals/fixtures/tiny-dashboard-stale-app/tests/dashboard-data.test.md +0 -9
- package/evals/fixtures/tiny-decision-destination-app/README.md +0 -6
- package/evals/fixtures/tiny-decision-destination-app/src/risk.ts +0 -8
- package/evals/fixtures/tiny-decision-destination-app/tests/risk.test.md +0 -4
- package/evals/fixtures/tiny-design-depth-app/README.md +0 -15
- package/evals/fixtures/tiny-design-depth-app/src/routes.ts +0 -25
- package/evals/fixtures/tiny-design-depth-app/src/sendEmail.ts +0 -3
- package/evals/fixtures/tiny-design-depth-app/src/sendSms.ts +0 -3
- package/evals/fixtures/tiny-design-depth-app/src/telemetry.ts +0 -3
- package/evals/fixtures/tiny-design-depth-app/tests/routes.test.md +0 -12
- package/evals/fixtures/tiny-eval-method-app/README.md +0 -8
- package/evals/fixtures/tiny-eval-method-app/evals/reports/review-pr-eval-result.md +0 -5
- package/evals/fixtures/tiny-eval-method-app/evals/reports/triage-question-failure.md +0 -17
- package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.diff +0 -11
- package/evals/fixtures/tiny-eval-method-app/evals/runs/review-pr-with-skill-output.md +0 -3
- package/evals/fixtures/tiny-eval-method-app/skills/review-pr/SKILL.md +0 -10
- package/evals/fixtures/tiny-eval-method-app/skills/triage-question/SKILL.md +0 -10
- package/evals/fixtures/tiny-execute-plan-app/README.md +0 -5
- package/evals/fixtures/tiny-execute-plan-app/checks/settings-and-billing-check.sh +0 -5
- package/evals/fixtures/tiny-execute-plan-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-execute-plan-app/docs/plans/immediate-downgrade-plan.md +0 -11
- package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-plan.md +0 -12
- package/evals/fixtures/tiny-execute-plan-app/docs/plans/settings-title-with-bad-check.md +0 -15
- package/evals/fixtures/tiny-execute-plan-app/docs/plans/unverified-billing-plan.md +0 -9
- package/evals/fixtures/tiny-execute-plan-app/docs/specs/failed-payment-grace-period.md +0 -9
- package/evals/fixtures/tiny-execute-plan-app/docs/specs/settings-title.md +0 -9
- package/evals/fixtures/tiny-execute-plan-app/src/billing.ts +0 -7
- package/evals/fixtures/tiny-execute-plan-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-execute-plan-app/tests/billing.test.md +0 -7
- package/evals/fixtures/tiny-execute-plan-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-execute-review-loop-app/README.md +0 -5
- package/evals/fixtures/tiny-execute-review-loop-app/docs/billing-policy.md +0 -9
- package/evals/fixtures/tiny-execute-review-loop-app/docs/plans/settings-title-plan.md +0 -12
- package/evals/fixtures/tiny-execute-review-loop-app/docs/review/previous-agent-summary.md +0 -15
- package/evals/fixtures/tiny-execute-review-loop-app/docs/review/review-pass-1-non-pass.md +0 -11
- package/evals/fixtures/tiny-execute-review-loop-app/docs/specs/settings-title.md +0 -9
- package/evals/fixtures/tiny-execute-review-loop-app/src/billing.ts +0 -7
- package/evals/fixtures/tiny-execute-review-loop-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-execute-review-loop-app/tests/billing.test.md +0 -7
- package/evals/fixtures/tiny-execute-review-loop-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-flaky-session-app/README.md +0 -10
- package/evals/fixtures/tiny-flaky-session-app/src/session-timeout.ts +0 -13
- package/evals/fixtures/tiny-flaky-session-app/tests/session-timeout.test.md +0 -8
- package/evals/fixtures/tiny-handoff-app/README.md +0 -5
- package/evals/fixtures/tiny-handoff-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-handoff-app/docs/handoffs/old-agent-handoff.md +0 -8
- package/evals/fixtures/tiny-handoff-app/docs/plans/billing-retry-plan.md +0 -11
- package/evals/fixtures/tiny-handoff-app/src/billing.ts +0 -7
- package/evals/fixtures/tiny-handoff-report-app/README.md +0 -3
- package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.diff +0 -1
- package/evals/fixtures/tiny-handoff-report-app/evals/runs/handoff-3/hof-003/with-skill-output.md +0 -1
- package/evals/fixtures/tiny-handoff-report-app/evals/scores/hof-003-score.txt +0 -4
- package/evals/fixtures/tiny-plan-app/README.md +0 -5
- package/evals/fixtures/tiny-plan-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-plan-app/docs/specs/failed-payment-grace-period.md +0 -25
- package/evals/fixtures/tiny-plan-app/src/billing.ts +0 -17
- package/evals/fixtures/tiny-plan-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-plan-app/tests/billing.test.md +0 -8
- package/evals/fixtures/tiny-plan-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-plan-strict-api-app/README.md +0 -3
- package/evals/fixtures/tiny-plan-strict-api-app/docs/billing-policy.md +0 -5
- package/evals/fixtures/tiny-plan-strict-api-app/docs/specs/billing-webhook-api.md +0 -25
- package/evals/fixtures/tiny-plan-strict-api-app/src/billing.ts +0 -5
- package/evals/fixtures/tiny-plan-strict-api-app/src/webhooks.ts +0 -3
- package/evals/fixtures/tiny-plan-strict-api-app/tests/billing-webhook.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-invalid-config-app/.freeflow/config.json +0 -3
- package/evals/fixtures/tiny-post-setup-invalid-config-app/AGENTS.md +0 -15
- package/evals/fixtures/tiny-post-setup-invalid-config-app/README.md +0 -16
- package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/auth-review.md +0 -15
- package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-post-setup-invalid-config-app/docs/copy-conventions.md +0 -15
- package/evals/fixtures/tiny-post-setup-invalid-config-app/src/auth-middleware.ts +0 -12
- package/evals/fixtures/tiny-post-setup-invalid-config-app/src/billing.ts +0 -20
- package/evals/fixtures/tiny-post-setup-invalid-config-app/src/login-copy.ts +0 -1
- package/evals/fixtures/tiny-post-setup-invalid-config-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/auth-middleware.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/billing.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-invalid-config-app/tests/copy.test.md +0 -7
- package/evals/fixtures/tiny-post-setup-missing-config-app/AGENTS.md +0 -15
- package/evals/fixtures/tiny-post-setup-missing-config-app/README.md +0 -16
- package/evals/fixtures/tiny-post-setup-missing-config-app/docs/auth-review.md +0 -15
- package/evals/fixtures/tiny-post-setup-missing-config-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-post-setup-missing-config-app/docs/copy-conventions.md +0 -15
- package/evals/fixtures/tiny-post-setup-missing-config-app/src/auth-middleware.ts +0 -12
- package/evals/fixtures/tiny-post-setup-missing-config-app/src/billing.ts +0 -20
- package/evals/fixtures/tiny-post-setup-missing-config-app/src/login-copy.ts +0 -1
- package/evals/fixtures/tiny-post-setup-missing-config-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-post-setup-missing-config-app/tests/auth-middleware.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-missing-config-app/tests/billing.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-missing-config-app/tests/copy.test.md +0 -7
- package/evals/fixtures/tiny-post-setup-source-conflict-app/.freeflow/config.json +0 -3
- package/evals/fixtures/tiny-post-setup-source-conflict-app/AGENTS.md +0 -15
- package/evals/fixtures/tiny-post-setup-source-conflict-app/README.md +0 -16
- package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/auth-review.md +0 -15
- package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-post-setup-source-conflict-app/docs/copy-conventions.md +0 -15
- package/evals/fixtures/tiny-post-setup-source-conflict-app/src/auth-middleware.ts +0 -12
- package/evals/fixtures/tiny-post-setup-source-conflict-app/src/billing.ts +0 -20
- package/evals/fixtures/tiny-post-setup-source-conflict-app/src/login-copy.ts +0 -1
- package/evals/fixtures/tiny-post-setup-source-conflict-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/auth-middleware.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/billing.test.md +0 -8
- package/evals/fixtures/tiny-post-setup-source-conflict-app/tests/copy.test.md +0 -7
- package/evals/fixtures/tiny-report-performance-app/README.md +0 -11
- package/evals/fixtures/tiny-report-performance-app/docs/performance.md +0 -10
- package/evals/fixtures/tiny-report-performance-app/src/report-summary.ts +0 -29
- package/evals/fixtures/tiny-report-performance-app/tests/report-performance.test.md +0 -8
- package/evals/fixtures/tiny-review-request-app/README.md +0 -3
- package/evals/fixtures/tiny-review-request-app/docs/billing-policy.md +0 -5
- package/evals/fixtures/tiny-review-request-app/docs/plans/billing-webhook-api.md +0 -19
- package/evals/fixtures/tiny-review-request-app/docs/review/previous-agent-summary.md +0 -20
- package/evals/fixtures/tiny-review-request-app/docs/specs/billing-webhook-api.md +0 -25
- package/evals/fixtures/tiny-review-request-app/src/billing.ts +0 -11
- package/evals/fixtures/tiny-review-request-app/src/webhooks.ts +0 -19
- package/evals/fixtures/tiny-review-request-app/tests/billing-webhook.test.md +0 -8
- package/evals/fixtures/tiny-review-work-app/README.md +0 -5
- package/evals/fixtures/tiny-review-work-app/docs/billing-policy.md +0 -9
- package/evals/fixtures/tiny-review-work-app/docs/plans/settings-title-plan.md +0 -12
- package/evals/fixtures/tiny-review-work-app/docs/review/previous-agent-summary.md +0 -15
- package/evals/fixtures/tiny-review-work-app/docs/specs/settings-title.md +0 -9
- package/evals/fixtures/tiny-review-work-app/src/billing.ts +0 -7
- package/evals/fixtures/tiny-review-work-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-review-work-app/tests/billing.test.md +0 -7
- package/evals/fixtures/tiny-review-work-app/tests/settings.test.md +0 -5
- package/evals/fixtures/tiny-saas-app/README.md +0 -16
- package/evals/fixtures/tiny-saas-app/docs/auth-review.md +0 -15
- package/evals/fixtures/tiny-saas-app/docs/billing-policy.md +0 -10
- package/evals/fixtures/tiny-saas-app/docs/copy-conventions.md +0 -15
- package/evals/fixtures/tiny-saas-app/docs/specs/settings-title.md +0 -13
- package/evals/fixtures/tiny-saas-app/src/auth-middleware.ts +0 -12
- package/evals/fixtures/tiny-saas-app/src/billing.ts +0 -20
- package/evals/fixtures/tiny-saas-app/src/login-copy.ts +0 -1
- package/evals/fixtures/tiny-saas-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-saas-app/tests/auth-middleware.test.md +0 -8
- package/evals/fixtures/tiny-saas-app/tests/billing.test.md +0 -8
- package/evals/fixtures/tiny-saas-app/tests/copy.test.md +0 -7
- package/evals/fixtures/tiny-setup-agents-app/AGENTS.md +0 -6
- package/evals/fixtures/tiny-setup-both-app/AGENTS.md +0 -5
- package/evals/fixtures/tiny-setup-both-app/CLAUDE.md +0 -5
- package/evals/fixtures/tiny-setup-claude-app/CLAUDE.md +0 -6
- package/evals/fixtures/tiny-setup-codex-rules-app/.codex/rules/shell.rules +0 -5
- package/evals/fixtures/tiny-setup-codex-rules-app/AGENTS.md +0 -6
- package/evals/fixtures/tiny-setup-conflict-app/AGENTS.md +0 -6
- package/evals/fixtures/tiny-setup-existing-block-app/AGENTS.md +0 -14
- package/evals/fixtures/tiny-setup-four-rules-app/CLAUDE.md +0 -6
- package/evals/fixtures/tiny-skill-dev-app/README.md +0 -10
- package/evals/fixtures/tiny-skill-dev-app/evals/prompts/review-pr-001.txt +0 -8
- package/evals/fixtures/tiny-skill-dev-app/evals/reports/review-pr-failure.md +0 -17
- package/evals/fixtures/tiny-skill-dev-app/skills/review-pr/SKILL.md +0 -13
- package/evals/fixtures/tiny-team-app/README.md +0 -5
- package/evals/fixtures/tiny-team-app/docs/product.md +0 -6
- package/evals/fixtures/tiny-team-app/src/teams.ts +0 -9
- package/evals/fixtures/tiny-team-app/src/users.ts +0 -5
- package/evals/fixtures/tiny-team-app/tests/teams.test.md +0 -7
- package/evals/fixtures/tiny-verify-app/README.md +0 -5
- package/evals/fixtures/tiny-verify-app/src/settings-nav.ts +0 -1
- package/evals/fixtures/tiny-verify-app/src/settings-page.ts +0 -1
- package/evals/fixtures/tiny-verify-app/tests/settings.test.md +0 -7
- package/evals/prompts/aon-001.txt +0 -2
- package/evals/prompts/byp-001.txt +0 -2
- package/evals/prompts/byp-002.txt +0 -2
- package/evals/prompts/cap-001.txt +0 -3
- package/evals/prompts/cap-002.txt +0 -3
- package/evals/prompts/cmd-001.txt +0 -3
- package/evals/prompts/cmd-002.txt +0 -3
- package/evals/prompts/cmd-003.txt +0 -6
- package/evals/prompts/cmd-004.txt +0 -5
- package/evals/prompts/cmd-005.txt +0 -5
- package/evals/prompts/cmd-006.txt +0 -5
- package/evals/prompts/cmd-007.txt +0 -5
- package/evals/prompts/cmd-008.txt +0 -5
- package/evals/prompts/cmd-009.txt +0 -8
- package/evals/prompts/cmd-010.txt +0 -5
- package/evals/prompts/cmd-011.txt +0 -5
- package/evals/prompts/cmd-012.txt +0 -5
- package/evals/prompts/cmd-013.txt +0 -5
- package/evals/prompts/cmd-014.txt +0 -7
- package/evals/prompts/cmd-015.txt +0 -7
- package/evals/prompts/cmd-016.txt +0 -5
- package/evals/prompts/cmt-001.txt +0 -1
- package/evals/prompts/cmt-002.txt +0 -1
- package/evals/prompts/cmt-003.txt +0 -1
- package/evals/prompts/cmt-004.txt +0 -1
- package/evals/prompts/cmt-005.txt +0 -1
- package/evals/prompts/dfd-001.txt +0 -3
- package/evals/prompts/dia-001.txt +0 -3
- package/evals/prompts/dia-002.txt +0 -3
- package/evals/prompts/dia-003.txt +0 -3
- package/evals/prompts/dis-001.txt +0 -5
- package/evals/prompts/dis-002.txt +0 -5
- package/evals/prompts/esk-001.txt +0 -5
- package/evals/prompts/esk-002.txt +0 -5
- package/evals/prompts/esk-003.txt +0 -5
- package/evals/prompts/grc-001.txt +0 -3
- package/evals/prompts/hof-001.txt +0 -3
- package/evals/prompts/hof-002.txt +0 -1
- package/evals/prompts/hof-003.txt +0 -1
- package/evals/prompts/hof-004.txt +0 -1
- package/evals/prompts/hof-005.txt +0 -1
- package/evals/prompts/ivg-001.txt +0 -1
- package/evals/prompts/ivg-002.txt +0 -11
- package/evals/prompts/ivg-003.txt +0 -1
- package/evals/prompts/ivg-004.txt +0 -1
- package/evals/prompts/mode-001.txt +0 -3
- package/evals/prompts/mode-002.txt +0 -1
- package/evals/prompts/mode-003.txt +0 -3
- package/evals/prompts/mode-004.txt +0 -3
- package/evals/prompts/mode-005.txt +0 -3
- package/evals/prompts/mode-006.txt +0 -3
- package/evals/prompts/mode-007.txt +0 -3
- package/evals/prompts/otr-001.txt +0 -3
- package/evals/prompts/otr-002.txt +0 -3
- package/evals/prompts/rar-001.txt +0 -5
- package/evals/prompts/rar-002.txt +0 -3
- package/evals/prompts/rar-003.txt +0 -6
- package/evals/prompts/rar-004.txt +0 -15
- package/evals/prompts/rar-005.txt +0 -10
- package/evals/prompts/rbr-001.txt +0 -3
- package/evals/prompts/rbr-002.txt +0 -3
- package/evals/prompts/rbr-003.txt +0 -7
- package/evals/prompts/rev-001.txt +0 -3
- package/evals/prompts/rev-002.txt +0 -5
- package/evals/prompts/rev-003.txt +0 -6
- package/evals/prompts/rev-004.txt +0 -1
- package/evals/prompts/rev-005.txt +0 -15
- package/evals/prompts/rev-006.txt +0 -5
- package/evals/prompts/stp-001.txt +0 -1
- package/evals/prompts/stp-002.txt +0 -1
- package/evals/prompts/stp-003.txt +0 -1
- package/evals/prompts/stp-004.txt +0 -1
- package/evals/prompts/stp-005.txt +0 -1
- package/evals/prompts/stp-006.txt +0 -1
- package/evals/prompts/stp-007.txt +0 -1
- package/evals/prompts/stp-008.txt +0 -1
- package/evals/prompts/stp-009.txt +0 -1
- package/evals/prompts/stp-010.txt +0 -1
- package/evals/prompts/stp-011.txt +0 -1
- package/evals/prompts/vfy-003.txt +0 -1
- package/evals/prompts/wpl-001.txt +0 -3
- package/evals/prompts/wpl-002.txt +0 -5
- package/evals/prompts/wpl-003.txt +0 -3
- package/evals/prompts/wpl-004.txt +0 -5
- package/evals/prompts/wpl-005.txt +0 -1
- package/evals/prompts/wrc-001.txt +0 -10
- package/evals/prompts/wrc-002.txt +0 -3
- package/evals/prompts/wsk-001.txt +0 -6
- package/evals/prompts/wsp-001.txt +0 -5
- package/evals/prompts/wsp-002.txt +0 -15
- package/evals/prompts/wsp-003.txt +0 -3
- package/evals/prompts/wsp-004.txt +0 -15
- package/evals/prompts/wsp-005.txt +0 -5
- package/evals/prompts/wsp-006.txt +0 -12
- package/evals/prompts/wsp-007.txt +0 -9
- package/evals/prompts/wsp-008.txt +0 -13
- package/evals/prompts/xpl-001.txt +0 -1
- package/evals/prompts/xpl-002.txt +0 -3
- package/evals/prompts/xpl-003.txt +0 -3
- package/evals/prompts/xpl-004.txt +0 -3
- package/evals/prompts/xpl-005.txt +0 -5
- package/evals/prompts/xpl-006.txt +0 -3
- package/evals/registries/adversarial-evals.json +0 -39
- package/evals/registries/fixture-evals.json +0 -2513
- package/evals/registries/handoff-evals.json +0 -74
- package/evals/registries/legacy-evals.json +0 -147
- package/evals/registries/skill-evidence.json +0 -193
- package/evals/registries/verify-work-evals.json +0 -46
- package/evals/reports/acceptance/v0.1-acceptance-report.md +0 -130
- package/evals/reports/by-command-surface/command-surface-1-report.md +0 -45
- package/evals/reports/by-command-surface/command-surface-10-report.md +0 -76
- package/evals/reports/by-command-surface/command-surface-11-report.md +0 -77
- package/evals/reports/by-command-surface/command-surface-12-report.md +0 -91
- package/evals/reports/by-command-surface/command-surface-13-report.md +0 -64
- package/evals/reports/by-command-surface/command-surface-14-report.md +0 -66
- package/evals/reports/by-command-surface/command-surface-15-report.md +0 -124
- package/evals/reports/by-command-surface/command-surface-16-report.md +0 -39
- package/evals/reports/by-command-surface/command-surface-2-report.md +0 -80
- package/evals/reports/by-command-surface/command-surface-3-report.md +0 -86
- package/evals/reports/by-command-surface/command-surface-4-report.md +0 -76
- package/evals/reports/by-command-surface/command-surface-5-report.md +0 -80
- package/evals/reports/by-command-surface/command-surface-6-report.md +0 -74
- package/evals/reports/by-command-surface/command-surface-7-report.md +0 -84
- package/evals/reports/by-command-surface/command-surface-8-report.md +0 -79
- package/evals/reports/by-command-surface/command-surface-9-report.md +0 -79
- package/evals/reports/by-command-surface/command-surface-matrix.md +0 -91
- package/evals/reports/by-skill/bypass-1-report.md +0 -124
- package/evals/reports/by-skill/capture-decisions-1-report.md +0 -99
- package/evals/reports/by-skill/capture-decisions-2-report.md +0 -102
- package/evals/reports/by-skill/commit-work-1-report.md +0 -165
- package/evals/reports/by-skill/commit-work-2-report.md +0 -109
- package/evals/reports/by-skill/design-for-depth-1-report.md +0 -95
- package/evals/reports/by-skill/diagnose-failure-1-report.md +0 -91
- package/evals/reports/by-skill/diagnose-failure-2-report.md +0 -94
- package/evals/reports/by-skill/discover-1-report.md +0 -118
- package/evals/reports/by-skill/evaluate-skill-2-report.md +0 -71
- package/evals/reports/by-skill/execute-plan-1-report.md +0 -103
- package/evals/reports/by-skill/execute-plan-2-report.md +0 -112
- package/evals/reports/by-skill/execute-plan-3-report.md +0 -87
- package/evals/reports/by-skill/grill-context-1-report.md +0 -103
- package/evals/reports/by-skill/handoff-1-report.md +0 -123
- package/evals/reports/by-skill/handoff-2-report.md +0 -105
- package/evals/reports/by-skill/handoff-3-report.md +0 -100
- package/evals/reports/by-skill/handoff-7-report.md +0 -77
- package/evals/reports/by-skill/interview-gate-1-report.md +0 -63
- package/evals/reports/by-skill/interview-gate-2-report.md +0 -105
- package/evals/reports/by-skill/mode-contract-1-report.md +0 -74
- package/evals/reports/by-skill/mode-contract-2-report.md +0 -87
- package/evals/reports/by-skill/mode-contract-3-report.md +0 -65
- package/evals/reports/by-skill/mode-contract-4-report.md +0 -71
- package/evals/reports/by-skill/mode-contract-5-report.md +0 -70
- package/evals/reports/by-skill/output-router-1-report.md +0 -76
- package/evals/reports/by-skill/output-router-2-report.md +0 -65
- package/evals/reports/by-skill/research-1-report.md +0 -144
- package/evals/reports/by-skill/research-brief-1-report.md +0 -94
- package/evals/reports/by-skill/research-brief-2-report.md +0 -75
- package/evals/reports/by-skill/review-artifact-1-report.md +0 -91
- package/evals/reports/by-skill/review-artifact-2-report.md +0 -79
- package/evals/reports/by-skill/review-artifact-3-report.md +0 -120
- package/evals/reports/by-skill/review-artifact-4-report.md +0 -89
- package/evals/reports/by-skill/review-artifact-5-report.md +0 -66
- package/evals/reports/by-skill/review-work-1-report.md +0 -85
- package/evals/reports/by-skill/review-work-2-report.md +0 -87
- package/evals/reports/by-skill/review-work-3-report.md +0 -104
- package/evals/reports/by-skill/review-work-4-report.md +0 -108
- package/evals/reports/by-skill/review-work-5-report.md +0 -73
- package/evals/reports/by-skill/review-work-6-report.md +0 -66
- package/evals/reports/by-skill/setup-freeflow-1-report.md +0 -199
- package/evals/reports/by-skill/setup-freeflow-2-report.md +0 -105
- package/evals/reports/by-skill/setup-freeflow-3-report.md +0 -105
- package/evals/reports/by-skill/setup-freeflow-4-report.md +0 -88
- package/evals/reports/by-skill/setup-freeflow-5-report.md +0 -115
- package/evals/reports/by-skill/skill-authoring-1-report.md +0 -81
- package/evals/reports/by-skill/verify-work-1-report.md +0 -111
- package/evals/reports/by-skill/verify-work-2-report.md +0 -120
- package/evals/reports/by-skill/workflow-1-report.md +0 -65
- package/evals/reports/by-skill/write-plan-1-report.md +0 -123
- package/evals/reports/by-skill/write-plan-2-report.md +0 -98
- package/evals/reports/by-skill/write-plan-3-report.md +0 -110
- package/evals/reports/by-skill/write-spec-1-report.md +0 -88
- package/evals/reports/by-skill/write-spec-2-report.md +0 -89
- package/evals/reports/by-skill/write-spec-3-report.md +0 -90
- package/evals/reports/by-skill/write-spec-4-report.md +0 -167
- package/evals/reports/harness/claude-harness-1-report.md +0 -103
- package/evals/reports/harness/eval-harness-1-report.md +0 -91
- package/evals/reports/iterations/iteration-1-report.md +0 -71
- package/evals/reports/iterations/iteration-2-report.md +0 -78
- package/evals/reports/iterations/iteration-3-report.md +0 -118
- package/evals/reports/iterations/iteration-4-report.md +0 -131
- package/evals/reports/iterations/iteration-5-report.md +0 -149
- package/evals/reports/iterations/iteration-6-report.md +0 -136
- package/evals/reports/runtime/always-on-runtime-1-report.md +0 -107
- package/evals/reports/runtime/context-mode-normalized-benchmark-1-report.md +0 -52
- package/evals/reports/runtime/context-mode-real-deep-baseline-1-report.md +0 -410
- package/evals/reports/runtime/context-mode-real-deep-final-slice-11-report.md +0 -203
- package/evals/reports/runtime/context-mode-real-deep-final-slice-11-review.md +0 -51
- package/evals/reports/runtime/context-saving-native-tools-baseline-1-report.md +0 -99
- package/evals/reports/runtime/eryx-python-compatibility-reprobe-1-report.md +0 -166
- package/evals/reports/runtime/eryx-python-proof-spike-1-report.md +0 -96
- package/evals/reports/runtime/eryx-python-proof-spike-2-report.md +0 -151
- package/evals/reports/runtime/eryx-python-script-derive-execution-1-report.md +0 -233
- package/evals/reports/runtime/jq-script-derive-execution-1-report.md +0 -215
- package/evals/reports/runtime/jq-wasm-proof-spike-1-report.md +0 -67
- package/evals/reports/runtime/output-router-benchmark-1-report.md +0 -81
- package/evals/reports/runtime/output-router-capture-derive-provider-eval-1-report.md +0 -35
- package/evals/reports/runtime/output-router-codex-large-index-sqlite-adhoc-report.md +0 -150
- package/evals/reports/runtime/output-router-codex-qa-benchmark-1-report.md +0 -46
- package/evals/reports/runtime/output-router-command-benchmark-1-report.md +0 -62
- package/evals/reports/runtime/output-router-index-benchmark-1-report.md +0 -64
- package/evals/reports/runtime/output-router-regression-1-report.md +0 -81
- package/evals/reports/runtime/output-router-transform-eval-1-report.md +0 -28
- package/evals/reports/runtime/pi-observed-routing-eval-1-report.md +0 -44
- package/evals/reports/runtime/quickjs-script-derive-execution-1-report.md +0 -159
- package/evals/reports/runtime/quickjs-wasi-proof-spike-1-report.md +0 -55
- package/evals/reports/runtime/render-format-row-protocol-1-report.md +0 -58
- package/evals/reports/runtime/script-sandbox-adapter-selection-review-1-report.md +0 -84
- package/evals/reports/runtime/script-sandbox-probe-resource-hardening-1-report.md +0 -51
- package/evals/reports/runtime/storage-policy-benchmark-1-report.md +0 -176
- package/evals/reports/runtime/vault-index-storage-spike-1-report.md +0 -39
- package/evals/reports/runtime/workflow-context-hook-1-report.md +0 -82
- package/evals/runbooks/adversarial-eval-runbook.md +0 -28
- package/evals/runbooks/fixture-eval-runbook.md +0 -137
- package/evals/runbooks/handoff-eval-runbook.md +0 -22
- package/evals/runbooks/verify-work-eval-runbook.md +0 -27
- package/evals/scripts/audit-command-surface.sh +0 -160
- package/evals/scripts/check-activation-contract.sh +0 -121
- package/evals/scripts/check-runtime-context-hook.sh +0 -183
- package/evals/scripts/grade-fixture-eval.sh +0 -446
- package/evals/scripts/run-claude-fixture-eval.sh +0 -26
- package/evals/scripts/run-codex-fixture-eval.sh +0 -26
- package/evals/scripts/run-eryx-python-proof-spike.js +0 -559
- package/evals/scripts/run-fixture-eval-by-id.sh +0 -23
- package/evals/scripts/run-fixture-eval.sh +0 -464
- package/evals/scripts/run-jq-wasm-proof-spike.js +0 -299
- package/evals/scripts/run-output-router-transform-eval.js +0 -115
- package/evals/scripts/run-pi-observed-routing-eval.js +0 -478
- package/evals/scripts/run-quickjs-wasi-proof-spike.js +0 -339
- package/evals/scripts/skill-evidence.sh +0 -316
- package/evals/scripts/validate-release-metadata.sh +0 -486
- package/evals/suites/v0.1-acceptance-suite.md +0 -131
- package/pi-extension/dist/host-producer-identification.js +0 -186
- package/pi-extension/dist/mcp-config.js +0 -30
- package/pi-extension/dist/native-safety-net.js +0 -209
- package/pi-extension/dist/observed-tool-routing.js +0 -114
- package/pi-extension/dist/renderers.js +0 -835
- package/pi-extension/dist/router-tools.js +0 -438
- package/pi-extension/dist/runtime-context.js +0 -288
- package/pi-extension/dist/schemas.js +0 -254
- package/pi-extension/dist/status.js +0 -449
- package/pi-extension/dist/utils.js +0 -467
- package/pi-extension/src/runtime-context.ts +0 -347
- package/pi-extension/src/status.ts +0 -505
- package/plugin-docs/README.md +0 -40
- package/plugin-docs/adr/0001-three-modes.md +0 -27
- package/plugin-docs/adr/0002-evals-before-hooks.md +0 -17
- package/plugin-docs/adr/0003-release-boundary.md +0 -21
- package/plugin-docs/adr/0004-discover-replaces-shallow-discovery-skills.md +0 -26
- package/plugin-docs/adr/README.md +0 -10
- package/plugin-docs/architecture.md +0 -89
- package/plugin-docs/output-router.md +0 -396
- package/plugin-docs/release-evidence.md +0 -77
- package/plugin-docs/skills.md +0 -44
- package/plugin-docs/workflow.md +0 -57
- package/skills/diagnose-failure/references/feedback-loop-catalog.md +0 -25
- package/skills/discover/ARTIFACT-DESTINATIONS.md +0 -58
- package/skills/discover/CHECKPOINTS.md +0 -74
- package/skills/discover/SKILL.md +0 -173
- package/skills/evaluate-skill/references/eval-patterns.md +0 -41
- package/skills/evaluate-skill/references/grading-priority.md +0 -33
- package/skills/execute-plan/SKILL.md +0 -133
- package/skills/execute-plan/references/execution-map.md +0 -122
- package/skills/interview-gate/SKILL.md +0 -90
- package/skills/setup-freeflow/references/output-router-setup.md +0 -102
- package/skills/workflow/references/workflow-map.md +0 -104
- /package/pi-extension/src/{mcp-config.ts → integrations/mcp-config.ts} +0 -0
package/plugin-docs/workflow.md
DELETED
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
# Workflow
|
|
2
|
-
|
|
3
|
-
Freeflow is a workflow layer, not a new agent. It helps the agent choose the right amount of process for the task.
|
|
4
|
-
|
|
5
|
-
## Modes
|
|
6
|
-
|
|
7
|
-
- `conversation`: non-mutating discussion, read-only exploration, and planning in chat; edits or other state-changing work require switching to `workflow` or `strict-workflow` first.
|
|
8
|
-
- `workflow`: default for consequential work; use the workflow spine and scale detail to risk.
|
|
9
|
-
- `strict-workflow`: high-risk or hard-to-reverse work with stronger gates.
|
|
10
|
-
|
|
11
|
-
Use strict-workflow for security, privacy, billing, public APIs, migrations, data loss, compatibility, permissions, deployment, or irreversible architecture.
|
|
12
|
-
|
|
13
|
-
## Map
|
|
14
|
-
|
|
15
|
-
```mermaid
|
|
16
|
-
flowchart LR
|
|
17
|
-
Request{request}
|
|
18
|
-
Talk[conversation<br/>answer directly]
|
|
19
|
-
Discover[discover<br/>checkpoint]
|
|
20
|
-
Spec[spec]
|
|
21
|
-
Plan[plan]
|
|
22
|
-
Build[build<br/>execute / diagnose]
|
|
23
|
-
Check[review + verify]
|
|
24
|
-
Close[commit / handoff]
|
|
25
|
-
|
|
26
|
-
Request -->|question| Talk
|
|
27
|
-
Request -->|work| Discover
|
|
28
|
-
Discover --> Spec --> Plan --> Build --> Check --> Close
|
|
29
|
-
Check -. new evidence or failed check .-> Discover
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
The map is orienting, not mandatory. Small reversible work can skip unnecessary artifacts and gates. Use `/discover` for the discovery loop.
|
|
33
|
-
|
|
34
|
-
## Backward Edge
|
|
35
|
-
|
|
36
|
-
Loop back when new evidence changes the path:
|
|
37
|
-
|
|
38
|
-
```mermaid
|
|
39
|
-
flowchart LR
|
|
40
|
-
Trigger{new evidence<br/>conflict<br/>failed check<br/>owner decision}
|
|
41
|
-
Decide{changes<br/>next action?}
|
|
42
|
-
Continue[continue]
|
|
43
|
-
Reenter[clarify / discover<br/>revise spec or plan<br/>diagnose / split / stop]
|
|
44
|
-
|
|
45
|
-
Trigger --> Decide
|
|
46
|
-
Decide -->|no| Continue
|
|
47
|
-
Decide -->|yes| Reenter
|
|
48
|
-
Reenter -. resume .-> Decide
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
The agent should not silently choose the backward destination when the choice changes product behavior, scope, compatibility, public APIs, security, privacy, billing, data loss, permissions, or irreversible architecture.
|
|
52
|
-
|
|
53
|
-
## Bypass
|
|
54
|
-
|
|
55
|
-
Bypass skips ceremony, not judgment.
|
|
56
|
-
|
|
57
|
-
Use `bypass` only to skip an unnecessary gate. It does not skip user-owned decisions, source-truth conflicts, risky checks, or fresh verification before completion claims.
|
|
@@ -1,25 +0,0 @@
|
|
|
1
|
-
# Feedback Loop Catalog
|
|
2
|
-
|
|
3
|
-
Pick the smallest loop that proves the user's failure. Prefer loops the agent can run repeatedly.
|
|
4
|
-
|
|
5
|
-
## Common Loops
|
|
6
|
-
|
|
7
|
-
- Failing test: best when an existing seam reaches the bug.
|
|
8
|
-
- Repro command: CLI, script, package task, or one exact command with fixture input.
|
|
9
|
-
- HTTP script: one request or short request sequence against a local service.
|
|
10
|
-
- Browser script: UI flow with DOM, console, and network assertions.
|
|
11
|
-
- Captured trace: HAR, log slice, event payload, seed, screenshot timestamp, core dump, or production sample replayed locally.
|
|
12
|
-
- Throwaway harness: minimal script around the real module when no test framework exists.
|
|
13
|
-
- Differential loop: same input through old versus new code, config, dataset, or dependency version.
|
|
14
|
-
- Instrumented run: targeted logs or counters at boundaries that distinguish hypotheses.
|
|
15
|
-
|
|
16
|
-
## Loop Quality
|
|
17
|
-
|
|
18
|
-
Improve the loop before fixing:
|
|
19
|
-
|
|
20
|
-
- Faster: remove unrelated startup and setup.
|
|
21
|
-
- Sharper: assert the exact symptom, not generic success.
|
|
22
|
-
- More deterministic: pin time, seed randomness, isolate network and filesystem.
|
|
23
|
-
- More representative: use the reported input shape and environment boundary.
|
|
24
|
-
|
|
25
|
-
If none is possible, stop and ask for the smallest missing artifact: command, logs, trace, failing input, screenshot with timestamp, environment access, or permission for temporary instrumentation.
|
|
@@ -1,58 +0,0 @@
|
|
|
1
|
-
# Artifact Destinations
|
|
2
|
-
|
|
3
|
-
Use this when a discovery checkpoint or decision must survive beyond chat.
|
|
4
|
-
|
|
5
|
-
## Destination Test
|
|
6
|
-
|
|
7
|
-
Before writing, answer:
|
|
8
|
-
|
|
9
|
-
- What needs to survive?
|
|
10
|
-
- Who will treat the destination as authority?
|
|
11
|
-
- Which existing artifact already owns this kind of truth?
|
|
12
|
-
- Would writing here create a new convention?
|
|
13
|
-
- Would a future agent know to look here?
|
|
14
|
-
|
|
15
|
-
If the owner is unclear, ask before writing.
|
|
16
|
-
|
|
17
|
-
## Common Destinations
|
|
18
|
-
|
|
19
|
-
| Discovery output | Prefer | Avoid |
|
|
20
|
-
| --- | --- | --- |
|
|
21
|
-
| Domain term or meaning | Existing `CONTEXT.md`, glossary, or domain doc | README fallback; creating domain memory without asking |
|
|
22
|
-
| Feature behavior, scope, acceptance | Existing or requested spec | ADR unless the ADR test passes |
|
|
23
|
-
| Future implementation path | Plan | Hidden plan inside a discovery note |
|
|
24
|
-
| Evidence summary before requirements settle | Existing discovery, research, or design notes; otherwise ask | Creating `docs/research/` by default |
|
|
25
|
-
| Immediate continuation state | Handoff | Durable decision memory |
|
|
26
|
-
| Hard-to-reverse surprising tradeoff from alternatives | ADR | Product note that hides rationale |
|
|
27
|
-
| Rejected approach | ADR if tradeoff-heavy; otherwise spec/decision note | Changelog or README by default |
|
|
28
|
-
| No durable value | Chat checkpoint | New file for ceremony |
|
|
29
|
-
|
|
30
|
-
## ADR Test
|
|
31
|
-
|
|
32
|
-
Use an ADR only when all are true:
|
|
33
|
-
|
|
34
|
-
- Hard to reverse.
|
|
35
|
-
- Surprising without context.
|
|
36
|
-
- Chosen from real alternatives.
|
|
37
|
-
|
|
38
|
-
Otherwise use the owning spec, decision note, domain doc, handoff, or chat.
|
|
39
|
-
|
|
40
|
-
## Stop Rules
|
|
41
|
-
|
|
42
|
-
Do not create a new `CONTEXT.md`, glossary, `docs/decisions/`, `docs/specs/`, `docs/research/`, or ADR folder just because the checkpoint needs memory.
|
|
43
|
-
|
|
44
|
-
A missing destination convention is a decision point.
|
|
45
|
-
|
|
46
|
-
Do not put volatile file lists, command output, temporary TODOs, or implementation inventory in durable decision memory. Put continuation state in a handoff or plan when needed.
|
|
47
|
-
|
|
48
|
-
## Question Shape
|
|
49
|
-
|
|
50
|
-
When destination is ambiguous, ask one direct question:
|
|
51
|
-
|
|
52
|
-
```text
|
|
53
|
-
I have a checkpoint worth preserving: [what].
|
|
54
|
-
The narrowest destination is [recommended artifact] because [why].
|
|
55
|
-
Should I save it there?
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
Do not write the artifact before the user chooses the destination.
|
|
@@ -1,74 +0,0 @@
|
|
|
1
|
-
# Discovery Checkpoints
|
|
2
|
-
|
|
3
|
-
A checkpoint is the output of discovery. It preserves the current understanding and names the next route.
|
|
4
|
-
|
|
5
|
-
Discovery does not end by silently writing a spec, plan, or implementation.
|
|
6
|
-
|
|
7
|
-
## Chat Or Artifact
|
|
8
|
-
|
|
9
|
-
Keep the checkpoint in chat when the work is short-lived and the next turn can safely continue from conversation context.
|
|
10
|
-
|
|
11
|
-
Save the checkpoint when it must guide later work, another agent, a future session, or a durable artifact.
|
|
12
|
-
|
|
13
|
-
Use the narrowest owning artifact:
|
|
14
|
-
|
|
15
|
-
- Spec: behavior, scope, requirements, acceptance, product decisions.
|
|
16
|
-
- Plan: future implementation path and checks.
|
|
17
|
-
- Handoff: immediate continuation state.
|
|
18
|
-
- Decision note or ADR: durable tradeoff or rejected approach.
|
|
19
|
-
- Domain memory: stable term or domain meaning.
|
|
20
|
-
- Discovery note: evidence summary that must be reused before requirements are settled.
|
|
21
|
-
|
|
22
|
-
If no clear destination exists, ask before writing.
|
|
23
|
-
|
|
24
|
-
## Shape
|
|
25
|
-
|
|
26
|
-
Use this shape. Omit sections that do not apply.
|
|
27
|
-
|
|
28
|
-
```text
|
|
29
|
-
Question / goal: ...
|
|
30
|
-
Evidence: ...
|
|
31
|
-
Current understanding: ...
|
|
32
|
-
Settled: ...
|
|
33
|
-
Tentative: ...
|
|
34
|
-
Open: ...
|
|
35
|
-
Recommendation: ...
|
|
36
|
-
Next: Forward | Backward | Branch | Stop — ...
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
## Good Checkpoints
|
|
40
|
-
|
|
41
|
-
A good checkpoint:
|
|
42
|
-
|
|
43
|
-
- Links or names live evidence instead of copying volatile inventory.
|
|
44
|
-
- Separates settled facts from tentative assumptions.
|
|
45
|
-
- Names open decisions that change the next route.
|
|
46
|
-
- Recommends the next route without treating it as permission.
|
|
47
|
-
- Avoids hiding user-owned decisions in polished prose.
|
|
48
|
-
|
|
49
|
-
## Bad Checkpoints
|
|
50
|
-
|
|
51
|
-
Bad:
|
|
52
|
-
|
|
53
|
-
```text
|
|
54
|
-
We discussed a lot. I'll implement the recommended path.
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
because it skips settled/tentative/open state and jumps to action.
|
|
58
|
-
|
|
59
|
-
Bad:
|
|
60
|
-
|
|
61
|
-
```text
|
|
62
|
-
Decision: create docs/decisions/ and record everything.
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
when the repo has no decision-note convention and the user has not chosen that destination.
|
|
66
|
-
|
|
67
|
-
Good:
|
|
68
|
-
|
|
69
|
-
```text
|
|
70
|
-
Settled: RiskReview is manual today; $500+ invoices enter review.
|
|
71
|
-
Tentative: retry scheduling should assist support, not replace review.
|
|
72
|
-
Open: should this be support-assistive or fully automated?
|
|
73
|
-
Next: Backward — answer the automation-scope decision before spec.
|
|
74
|
-
```
|
package/skills/discover/SKILL.md
DELETED
|
@@ -1,173 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: discover
|
|
3
|
-
description: Use before the first answer or repo/code exploration when the user is shaping an idea, feature, tool/API surface, runtime behavior, architecture direction, vague task, bug-like unknown, or consequential hypothesis/question before spec, plan, build, or durable memory. Trigger especially on "should we", "why not", "what do you think", or design questions.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Discover
|
|
7
|
-
|
|
8
|
-
Discover is the context-building loop before action: evidence-guided discussion that finds the next high-leverage decision and ends in a checkpoint.
|
|
9
|
-
|
|
10
|
-
If this skill matches, load it before the first repo search, code read, web lookup, command, design answer, or artifact. Direct questions still use discovery-light: inspect the smallest relevant evidence, then answer directly.
|
|
11
|
-
|
|
12
|
-
Priority: Interview Gate stops silent decisions and user-owned decisions first. Discover handles context-building after no immediate stop condition remains. Output Router only chooses evidence transport after the workflow/discovery route is clear.
|
|
13
|
-
|
|
14
|
-
Do not turn discovery into a questionnaire, a report by default, or a decision-capture script. The hard part is shared understanding, not filling an artifact.
|
|
15
|
-
|
|
16
|
-
For short-lived work, keep the checkpoint in chat. When it must guide later work, save it in the narrowest owning artifact: spec, plan, handoff, decision note, ADR, domain memory, or discovery note.
|
|
17
|
-
|
|
18
|
-
Goal: enough evidence and shared understanding to route to spec, plan, more discovery, durable memory, build, or stop without inventing product, domain, scope, public API, security, privacy, billing, data-loss, compatibility, or architecture decisions.
|
|
19
|
-
|
|
20
|
-
Hard stop: do not satisfy requests for “every question,” “grill me,” “25-50 questions,” or recommended answers for each question. That creates an approval script. Inspect evidence, name the decision tree, and ask the next one or two path-changing questions instead.
|
|
21
|
-
|
|
22
|
-
## Evidence Before Questions
|
|
23
|
-
|
|
24
|
-
Inspect first when the answer is discoverable:
|
|
25
|
-
|
|
26
|
-
- code, tests, docs, ADRs, specs, policies, issues, logs, traces, or current repo state;
|
|
27
|
-
- provided files, URLs, screenshots, transcripts, or user context;
|
|
28
|
-
- current external docs or web sources when framework/library facts, ecosystem behavior, or recent changes matter.
|
|
29
|
-
|
|
30
|
-
Ask before inspecting only when multiple targets would change the discovery path or when access/scope is user-owned.
|
|
31
|
-
|
|
32
|
-
For a direct question, keep the loop light:
|
|
33
|
-
|
|
34
|
-
```text
|
|
35
|
-
question -> inspect smallest relevant evidence -> answer directly
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
No artifact, plan, or checkpoint is required unless the answer changes future consequential work.
|
|
39
|
-
|
|
40
|
-
## Design Pressure
|
|
41
|
-
|
|
42
|
-
Use `../design-for-depth/SKILL.md` when discovery shapes modules, interfaces, seams, adapters, architecture direction, growing scope, codebase exploration, or repeated review/execution churn.
|
|
43
|
-
|
|
44
|
-
Do not decide architecture up front just because the lens is active. Use it to notice whether complexity is being hidden behind useful interfaces or spread across callers, tests, docs, artifacts, and future agents.
|
|
45
|
-
|
|
46
|
-
Use the deletion test on questions and decisions: if skipping it makes complexity spread downstream, it matters. If skipping it makes complexity vanish, it was ceremony.
|
|
47
|
-
|
|
48
|
-
## Scope Framing
|
|
49
|
-
|
|
50
|
-
Preserve the user's intended scope. Do not silently turn a large idea into `v1`/`v2`, MVP, release, or roadmap framing.
|
|
51
|
-
|
|
52
|
-
Use implementation slices, implementation phases, checklists, or multiple artifacts to make large work understandable and executable without changing what the user asked for.
|
|
53
|
-
|
|
54
|
-
Versioned shipping is a product/scope decision. Use it only when the user asks for release planning, the source context already uses that framing, or you have named why release boundaries matter and the user agrees.
|
|
55
|
-
|
|
56
|
-
If the full scope is too large for one safe pass, say that directly. Recommend implementation slices or artifact splits first; recommend versioned releases only when that is the user's actual problem.
|
|
57
|
-
|
|
58
|
-
## Loop
|
|
59
|
-
|
|
60
|
-
1. Identify the target: repo area, artifact, URL, product, model, package, paper, issue, symptom, or idea. If multiple plausible targets would change discovery, ask for the source.
|
|
61
|
-
2. Inspect the smallest evidence that can change the current route.
|
|
62
|
-
3. Explain the current understanding in plain language.
|
|
63
|
-
4. Brainstorm or compare paths when the direction is still forming; do not force a decision too early.
|
|
64
|
-
5. Ask one path-changing question when the answer would change the next artifact or action.
|
|
65
|
-
6. Let the user challenge, counter-question, or add constraints. Treat that as discovery input, not interruption.
|
|
66
|
-
7. Refresh evidence when new claims, conflicts, or branches appear.
|
|
67
|
-
8. Track decisions as settled, tentative, or open.
|
|
68
|
-
9. Stop when remaining ambiguity would not change the next route.
|
|
69
|
-
|
|
70
|
-
Discovery may move through evidence, brainstorming, targeted questioning, design, and checkpointing in any order. Do not force a fixed sequence when the evidence or conversation needs a loop.
|
|
71
|
-
|
|
72
|
-
## Final Question Pass
|
|
73
|
-
|
|
74
|
-
Before routing from discovery to `write-spec`, run a final ambiguity sweep.
|
|
75
|
-
|
|
76
|
-
Trigger it when:
|
|
77
|
-
|
|
78
|
-
- the user asks to write the spec;
|
|
79
|
-
- the user says brainstorming, discovery, or discussion feels complete;
|
|
80
|
-
- the user asks to move to the next phase;
|
|
81
|
-
- you are about to close discovery with `Next: Forward — write-spec`.
|
|
82
|
-
|
|
83
|
-
Ask the smallest set of remaining path-changing questions whose answers would affect spec scope, product behavior, acceptance, public API, architecture, security, privacy, billing, data-loss, compatibility, or artifact destination.
|
|
84
|
-
|
|
85
|
-
If meaningful ambiguity remains, route with `Next: Forward — final question pass before write-spec` and ask those questions. If no meaningful questions remain, say so and route to `write-spec`.
|
|
86
|
-
|
|
87
|
-
Do not use this while brainstorming, research, or comparison is still active. Do not turn it into a long questionnaire or recommendation approval script.
|
|
88
|
-
|
|
89
|
-
## Question Discipline
|
|
90
|
-
|
|
91
|
-
Prefer deeper questions over more questions. Ask as many as the situation needs, but each question should earn its place.
|
|
92
|
-
|
|
93
|
-
Before asking, run the deletion test:
|
|
94
|
-
|
|
95
|
-
```text
|
|
96
|
-
If I do not ask this, does downstream complexity spread?
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
If no, skip it.
|
|
100
|
-
|
|
101
|
-
Bad question:
|
|
102
|
-
|
|
103
|
-
```text
|
|
104
|
-
Should the button be blue or green?
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
when the feature purpose is still unclear.
|
|
108
|
-
|
|
109
|
-
Good question:
|
|
110
|
-
|
|
111
|
-
```text
|
|
112
|
-
Is this for one workflow or many workflows?
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
because it changes the module interface, spec shape, and test surface.
|
|
116
|
-
|
|
117
|
-
Long question chains are a failure mode, not diligence. Never number out a long list just because the user asked for one.
|
|
118
|
-
|
|
119
|
-
Do not turn recommendations into an approval script. Recommendations should clarify tradeoffs, not make it easy to rubber-stamp decisions. Give the recommendation for the current path-changing question, then wait when the choice is user-owned.
|
|
120
|
-
|
|
121
|
-
After a few path-changing questions, or whenever the user introduces a new claim, inspect evidence, summarize, or checkpoint before asking more.
|
|
122
|
-
|
|
123
|
-
## Decision Discipline
|
|
124
|
-
|
|
125
|
-
Read `ARTIFACT-DESTINATIONS.md` when a checkpoint or decision must survive beyond chat and the owning destination is not obvious.
|
|
126
|
-
|
|
127
|
-
Not every answer is durable. Keep a lightweight ledger:
|
|
128
|
-
|
|
129
|
-
- **Settled** — explicit user decision or evidence-backed fact.
|
|
130
|
-
- **Tentative** — useful working assumption, not authority.
|
|
131
|
-
- **Open** — unresolved question that can change the next route.
|
|
132
|
-
|
|
133
|
-
Persist a decision only when absence would make future work rediscover or contradict it.
|
|
134
|
-
|
|
135
|
-
Put durable decisions where they belong:
|
|
136
|
-
|
|
137
|
-
- Domain term or meaning -> existing glossary, `CONTEXT.md`, or domain doc.
|
|
138
|
-
- Feature behavior, scope, or acceptance -> spec.
|
|
139
|
-
- Future implementation path -> plan.
|
|
140
|
-
- Immediate continuation state -> handoff.
|
|
141
|
-
- Hard-to-reverse, surprising tradeoff chosen from alternatives -> ADR.
|
|
142
|
-
- No durable value -> leave it in chat.
|
|
143
|
-
|
|
144
|
-
If the destination is missing or ambiguous, ask before writing.
|
|
145
|
-
|
|
146
|
-
## Stop Conditions
|
|
147
|
-
|
|
148
|
-
Stop and ask when discovery reveals a user-owned decision, source-of-truth conflict, path conflict, material method substitution, or ambiguous artifact destination.
|
|
149
|
-
|
|
150
|
-
Do not write implementation files from discovery.
|
|
151
|
-
|
|
152
|
-
Write specs, plans, ADRs, handoffs, or decision notes only when the user asks or the checkpoint must guide later work.
|
|
153
|
-
|
|
154
|
-
Do not leave discovery by silently jumping to spec, plan, or build. Before `write-spec`, run the final question pass. Close with a checkpoint when consequential direction changed.
|
|
155
|
-
|
|
156
|
-
## Checkpoint Shape
|
|
157
|
-
|
|
158
|
-
Read `CHECKPOINTS.md` when saving a checkpoint or choosing chat versus artifact.
|
|
159
|
-
|
|
160
|
-
Use this shape when ending a discovery pass or routing forward. Save it in the owning artifact when it must survive beyond chat. Omit sections that do not apply.
|
|
161
|
-
|
|
162
|
-
```text
|
|
163
|
-
Question / goal: ...
|
|
164
|
-
Evidence: ...
|
|
165
|
-
Current understanding: ...
|
|
166
|
-
Settled: ...
|
|
167
|
-
Tentative: ...
|
|
168
|
-
Open: ...
|
|
169
|
-
Recommendation: ...
|
|
170
|
-
Next: Forward | Backward | Branch | Stop — ...
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
`Next:` is routing, not permission to create the next artifact.
|
|
@@ -1,41 +0,0 @@
|
|
|
1
|
-
# Eval Patterns
|
|
2
|
-
|
|
3
|
-
Choose the smallest repeatable artifact that preserves the failure and can be graded later.
|
|
4
|
-
|
|
5
|
-
## Common Shapes
|
|
6
|
-
|
|
7
|
-
Use a prompt/pass-criteria update when the failure is already captured and the missing piece is expected behavior.
|
|
8
|
-
|
|
9
|
-
Use a transcript note when the failure is conversational: answering vs artifact creation, clarification, refusal, routing, or command interpretation.
|
|
10
|
-
|
|
11
|
-
Use a fixture eval when files, repo evidence, commands, state files, installed memory, or generated artifacts affect the outcome.
|
|
12
|
-
|
|
13
|
-
Use saved-run grading when the job is to judge existing outputs, diffs, logs, or transcripts. Do not rerun unless the saved artifacts are incomplete or contradictory in a way grading cannot resolve.
|
|
14
|
-
|
|
15
|
-
Use separate baseline and with-skill fixtures when the tested behavior depends on installed memory, setup output, host config, or pre-existing state.
|
|
16
|
-
|
|
17
|
-
## Comparison Choices
|
|
18
|
-
|
|
19
|
-
Use no-skill baseline versus with-skill when proving the skill adds value over the ordinary agent or installed memory-free behavior.
|
|
20
|
-
|
|
21
|
-
Use previous skill version versus updated skill when testing a wording change, runtime context change, or regression fix for a skill that already exists.
|
|
22
|
-
|
|
23
|
-
For previous-version comparisons, save the exact old skill source, commit hash, plugin version, or temp copy path used as the control. Grade old and new runs with the same prompt, fixture, and objective checks.
|
|
24
|
-
|
|
25
|
-
## Harness Discipline
|
|
26
|
-
|
|
27
|
-
Use the repo's runner when one exists. Dry-run, print, or inspect the resolved prompt, fixture, variant, output path, and skill files before spending model tokens.
|
|
28
|
-
|
|
29
|
-
Do not build a harness because the user says "evaluate." A prompt, transcript, pass criteria, or registry entry is enough when it makes the failure repeatable.
|
|
30
|
-
|
|
31
|
-
"Do not add a harness" means avoid machinery. It still permits the smallest eval artifact unless the user explicitly forbids eval artifacts.
|
|
32
|
-
|
|
33
|
-
Save the final response and diff for fixture runs. Save command output or git status when those are the grading surface.
|
|
34
|
-
|
|
35
|
-
## Weak Eval Signs
|
|
36
|
-
|
|
37
|
-
- Baseline and with-skill both pass without meaningful behavioral difference.
|
|
38
|
-
- The prompt tells the agent the intended answer instead of creating pressure.
|
|
39
|
-
- The only grade is the agent's self-assessment.
|
|
40
|
-
- The eval cannot show whether files changed.
|
|
41
|
-
- The fixture forbids behavior the repo actually allows.
|
|
@@ -1,33 +0,0 @@
|
|
|
1
|
-
# Grading Priority
|
|
2
|
-
|
|
3
|
-
Grade artifacts before explanations.
|
|
4
|
-
|
|
5
|
-
Evidence order:
|
|
6
|
-
|
|
7
|
-
1. Filesystem diff, git diff, created/deleted files, and git state.
|
|
8
|
-
2. Command output, exit codes, logs, traces, screenshots, or saved reports.
|
|
9
|
-
3. Final response.
|
|
10
|
-
4. Full transcript.
|
|
11
|
-
5. Agent self-assessment.
|
|
12
|
-
|
|
13
|
-
If lower-priority evidence conflicts with higher-priority evidence, the higher-priority evidence wins.
|
|
14
|
-
|
|
15
|
-
## Pass And Fail Rules
|
|
16
|
-
|
|
17
|
-
A run passes only when the artifacts satisfy the eval assertions. A polished final response cannot repair a bad diff.
|
|
18
|
-
|
|
19
|
-
If a final response claims an eval artifact was created but the diff shows only a skill edit, grade it as missing eval-first evidence.
|
|
20
|
-
|
|
21
|
-
If the diff shows a prohibited file edit, grade the run by the diff even when the final response says no files changed.
|
|
22
|
-
|
|
23
|
-
If the outcome can be checked by line count, file existence, JSON parsing, git status, or diff content, check it mechanically.
|
|
24
|
-
|
|
25
|
-
Use transcript review only when the behavior cannot be judged from saved files, command output, or final response.
|
|
26
|
-
|
|
27
|
-
## Rerun Rules
|
|
28
|
-
|
|
29
|
-
Rerun the failed side first after changing skill wording.
|
|
30
|
-
|
|
31
|
-
Rerun both sides only when the prompt, fixture, harness, or grading criteria changed enough to invalidate the comparison.
|
|
32
|
-
|
|
33
|
-
Do not rerun to make a failing saved artifact look better. Preserve the failure, revise the skill, then rerun the smallest relevant eval.
|
|
@@ -1,133 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: execute-plan
|
|
3
|
-
description: Use when implementing an approved plan, executing planned slices, resuming planned work, handling planned verification failures, adjudicating review findings during execution, or encountering scope/source conflicts while carrying out a plan.
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Execute Plan
|
|
7
|
-
|
|
8
|
-
Execute the plan. Do not improve it silently.
|
|
9
|
-
|
|
10
|
-
The plan is instructions, not authority. Live repo evidence and source truth win.
|
|
11
|
-
|
|
12
|
-
Move one verified slice at a time. Preserve rollback, reviewability, and user control.
|
|
13
|
-
|
|
14
|
-
Read `references/execution-map.md` when the plan has multiple slices, TDD/benchmark work, review checkpoints, per-slice commits, context-window pressure, or any failed check/review/source-scope conflict.
|
|
15
|
-
|
|
16
|
-
Use `../design-for-depth/SKILL.md` when a slice changes modules, interfaces, seams, adapters, architecture, or when review findings become a stream of edge-case patches. Ask whether the slice is deepening the module or spreading complexity across callers, tests, docs, or review comments.
|
|
17
|
-
|
|
18
|
-
## Classify First
|
|
19
|
-
|
|
20
|
-
- Valid plan: inspect source context, execute the next slice, verify it.
|
|
21
|
-
- Plan/source conflict: stop and ask which source should change.
|
|
22
|
-
- Hidden owner decision: stop and ask.
|
|
23
|
-
- Missing verification: stop before consequential edits and ask to revise the plan or approve a diagnostic path.
|
|
24
|
-
- Missing plan: ask for a plan or route to `write-plan`.
|
|
25
|
-
- Scope expansion: stop before absorbing it; route back to discover, spec, or plan.
|
|
26
|
-
- Review failure during execution: classify findings and report the route before editing from them.
|
|
27
|
-
|
|
28
|
-
## Before Editing
|
|
29
|
-
|
|
30
|
-
Read:
|
|
31
|
-
|
|
32
|
-
- The plan.
|
|
33
|
-
- The source spec or requirements the plan cites.
|
|
34
|
-
- Relevant docs, tests, policies, ADRs, and code.
|
|
35
|
-
- Handoffs only as memory.
|
|
36
|
-
|
|
37
|
-
Live repo evidence overrides stale plans and handoffs.
|
|
38
|
-
|
|
39
|
-
Before each non-trivial slice, name the slice contract:
|
|
40
|
-
|
|
41
|
-
```text
|
|
42
|
-
Slice:
|
|
43
|
-
Source truth:
|
|
44
|
-
Module/interface changed:
|
|
45
|
-
Behavior/test/benchmark:
|
|
46
|
-
Verification:
|
|
47
|
-
Review checkpoint:
|
|
48
|
-
Commit or handoff checkpoint:
|
|
49
|
-
Stop conditions:
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Do not start a slice if the remaining context is not enough to orient, edit, verify, and checkpoint it. Stop with a handoff route instead.
|
|
53
|
-
|
|
54
|
-
Do not execute a plan that would:
|
|
55
|
-
|
|
56
|
-
- Invent or change product behavior, scope, domain meaning, compatibility, public API behavior, security, privacy, billing, data-loss, or architecture.
|
|
57
|
-
- Override docs, tests, specs, policies, ADRs, or established behavior.
|
|
58
|
-
- Treat "do not ask", "just execute", "latest context", or "handoff says" as conflict approval.
|
|
59
|
-
- Skip verification for consequential behavior.
|
|
60
|
-
- Rewrite a verification script, test, source-truth doc, policy, or spec to make the plan pass unless the plan explicitly authorized that artifact change and it matches source truth.
|
|
61
|
-
|
|
62
|
-
Name the conflict or missing decision. Ask which path to follow. Recommend the path supported by evidence.
|
|
63
|
-
|
|
64
|
-
For source-truth conflicts or missing verification, the final line must be a direct choice question.
|
|
65
|
-
|
|
66
|
-
For missing verification, ask whether to revise the plan to add a check or approve a specific verification path.
|
|
67
|
-
|
|
68
|
-
## Slice Execution
|
|
69
|
-
|
|
70
|
-
Work in vertical slices.
|
|
71
|
-
|
|
72
|
-
For each slice:
|
|
73
|
-
|
|
74
|
-
- Make only the edits needed for that slice.
|
|
75
|
-
- Keep the slice boundary intact; do not blend future slices into the current one for convenience.
|
|
76
|
-
- Run the planned check or the smallest equivalent check.
|
|
77
|
-
- If the check fails, stop and report the evidence before changing direction.
|
|
78
|
-
- If new evidence invalidates the plan, stop before patching forward.
|
|
79
|
-
- If implementation spreads policy, edge cases, or verification across callers/tests/docs, treat it as design pressure before adding more patches.
|
|
80
|
-
|
|
81
|
-
When TDD is requested by the user, plan, or repo practice, use TDD inside the slice:
|
|
82
|
-
|
|
83
|
-
```text
|
|
84
|
-
one behavior test or benchmark
|
|
85
|
-
-> minimal implementation
|
|
86
|
-
-> refactor while green
|
|
87
|
-
-> verify the slice
|
|
88
|
-
```
|
|
89
|
-
|
|
90
|
-
Do not write all tests first and all implementation later. Do not anticipate future slices.
|
|
91
|
-
|
|
92
|
-
After a planned verification command fails, do not edit the verifier, tests, docs, policy, source-truth files, or unrelated code to make it pass. This is true even when the verifier appears wrong and even when the user said to "fix whatever is needed". A bad verifier is a plan defect; stop, report the failing command and conflicting evidence, and ask whether to revise the plan or change the source truth.
|
|
93
|
-
|
|
94
|
-
Local reversible implementation details can be chosen from repo conventions.
|
|
95
|
-
|
|
96
|
-
If per-slice commits are requested, do not continue to the next slice with a verified slice still uncommitted. Route to `commit-work` after review/verification evidence exists.
|
|
97
|
-
|
|
98
|
-
## Review Checkpoints
|
|
99
|
-
|
|
100
|
-
Use review where the plan, risk, or slice boundary calls for it. Do not review every slice by habit.
|
|
101
|
-
|
|
102
|
-
A non-passing review is a phase exit, not an autonomous patch loop.
|
|
103
|
-
|
|
104
|
-
When a review returns findings during execution:
|
|
105
|
-
|
|
106
|
-
1. Inspect the relevant code, tests, docs, plan, and source truth.
|
|
107
|
-
2. Classify each material finding: accepted, rejected, question, or needs evidence.
|
|
108
|
-
3. Report the route before editing from that review batch.
|
|
109
|
-
|
|
110
|
-
The turn that receives a non-passing review ends with adjudication and route only. Do not edit from that review batch in the same turn, even when the user or reviewer says to apply all findings and continue reviewing.
|
|
111
|
-
|
|
112
|
-
Do not immediately apply findings and request another review in the same loop. Do not treat non-blocking findings or reviewer questions as automatic failure.
|
|
113
|
-
|
|
114
|
-
If all accepted findings are small, in-scope, and supported by source truth, recommend a bounded fix pass as the next route. Do not perform that fix pass until the user or parent explicitly chooses it after seeing the adjudication. If findings change scope, behavior, source truth, public API, security, privacy, billing, data loss, compatibility, or architecture, route backward before editing.
|
|
115
|
-
|
|
116
|
-
Three review passes is the hard cap for the same slice/work scope. At pass 3, stop, classify, and diagnose. Do not request a fourth broad review. Use a `diagnose-failure`-style loop to decide whether the issue is shallow discovery, weak spec, wrong plan slice, ambiguous policy, source-truth conflict, shallow module/interface, implementation bug, or stale reviewer context.
|
|
117
|
-
|
|
118
|
-
## Backward Edge
|
|
119
|
-
|
|
120
|
-
If implementation reveals that the plan is wrong, incomplete, or too narrow, re-enter discover/spec/plan before continuing.
|
|
121
|
-
|
|
122
|
-
Do not absorb material scope expansion into execution. New behavior, API shape, security/privacy/billing/data-loss decisions, compatibility changes, or irreversible architecture usually need a revised spec or plan.
|
|
123
|
-
|
|
124
|
-
## Completion
|
|
125
|
-
|
|
126
|
-
Report:
|
|
127
|
-
|
|
128
|
-
- What changed.
|
|
129
|
-
- What was verified.
|
|
130
|
-
- Review status and any adjudicated findings.
|
|
131
|
-
- Commit or handoff status when relevant.
|
|
132
|
-
- What remains unverified.
|
|
133
|
-
- Any blocked decisions or plan changes needed.
|