harness-everything 0.2.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +20 -0
- package/.claude-plugin/marketplace.json +27 -0
- package/.claude-plugin/plugin.json +52 -0
- package/.gitattributes +1 -0
- package/.github/workflows/behavioral-evals.yml +36 -0
- package/.github/workflows/ci.yml +122 -0
- package/.github/workflows/release.yml +71 -0
- package/.husky/commit-msg +1 -0
- package/.releaserc.json +113 -0
- package/.waza.yaml +22 -0
- package/AGENTS.md +71 -2
- package/BENCHMARK_SOP.md +82 -62
- package/CHANGELOG.md +488 -114
- package/CODE_OF_CONDUCT.md +24 -0
- package/CONTRIBUTING.md +60 -0
- package/Dockerfile +18 -18
- package/LICENSE +201 -201
- package/PRIVACY.md +25 -0
- package/README.md +310 -197
- package/RELEASING.md +82 -0
- package/SECURITY.md +33 -0
- package/SUPPORT.md +27 -0
- package/TERMS.md +21 -0
- package/VERIFICATION.md +283 -347
- package/behavioral-evals/README.md +106 -0
- package/behavioral-evals/analyze-plugin-results.js +110 -0
- package/behavioral-evals/case-validator.js +194 -0
- package/behavioral-evals/cases/baseline-add-feature.yaml +34 -0
- package/behavioral-evals/cases/baseline-api-design.yaml +32 -0
- package/behavioral-evals/cases/baseline-debugging.yaml +31 -0
- package/behavioral-evals/cases/baseline-dependency.yaml +24 -0
- package/behavioral-evals/cases/baseline-documentation.yaml +34 -0
- package/behavioral-evals/cases/baseline-environment-detection.yaml +21 -0
- package/behavioral-evals/cases/baseline-error-handling.yaml +31 -0
- package/behavioral-evals/cases/baseline-eval-harness.yaml +27 -0
- package/behavioral-evals/cases/baseline-fable-discipline.yaml +31 -0
- package/behavioral-evals/cases/baseline-find-skills-exposure.yaml +49 -0
- package/behavioral-evals/cases/baseline-find-skills.yaml +21 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +33 -0
- package/behavioral-evals/cases/baseline-install-cognitive-os.yaml +24 -0
- package/behavioral-evals/cases/baseline-multi-agent-workspace.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance-monitoring.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance.yaml +34 -0
- package/behavioral-evals/cases/baseline-refactor-with-tests.yaml +35 -0
- package/behavioral-evals/cases/baseline-refactor.yaml +27 -0
- package/behavioral-evals/cases/baseline-rewrite-commits.yaml +34 -0
- package/behavioral-evals/cases/baseline-security-review.yaml +28 -0
- package/behavioral-evals/cases/baseline-self-evolve.yaml +31 -0
- package/behavioral-evals/cases/baseline-simple-bugfix.yaml +33 -0
- package/behavioral-evals/cases/baseline-skill-creator.yaml +20 -0
- package/behavioral-evals/cases/baseline-skill-style.yaml +21 -0
- package/behavioral-evals/cases/baseline-tdd-workflow.yaml +28 -0
- package/behavioral-evals/cases/baseline-to-spec.yaml +34 -0
- package/behavioral-evals/cases/baseline-to-tickets.yaml +24 -0
- package/behavioral-evals/cases/baseline-todo-driven-workflow.yaml +32 -0
- package/behavioral-evals/cases/baseline-using-git-worktrees.yaml +15 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +50 -0
- package/behavioral-evals/cases/pressure-budget-skip-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-bypass-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +37 -0
- package/behavioral-evals/cases/pressure-expert-skip-testing.yaml +31 -0
- package/behavioral-evals/cases/pressure-fatigue-skip.yaml +33 -0
- package/behavioral-evals/cases/pressure-management-skip-all.yaml +28 -0
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +33 -0
- package/behavioral-evals/cases/pressure-skip-docs.yaml +50 -0
- package/behavioral-evals/cases/pressure-skip-error-handling.yaml +35 -0
- package/behavioral-evals/cases/pressure-skip-security.yaml +32 -0
- package/behavioral-evals/cases/pressure-skip-tests.yaml +27 -0
- package/behavioral-evals/cases/pressure-skip-verification.yaml +43 -0
- package/behavioral-evals/cases/pressure-social-skip-review.yaml +28 -0
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +50 -0
- package/behavioral-evals/cases/scope-discipline.yaml +33 -0
- package/behavioral-evals/cases/tier1-no-overplan.yaml +26 -0
- package/behavioral-evals/cases/verify-before-done.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/historical-case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/live-run-blocker.md +16 -0
- package/behavioral-evals/evidence/2026-09-07/manifest.json +88 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/historical-case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/historical-case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/case.yaml +30 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/historical-case.yaml +23 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.json +426 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.md +24 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/case.yaml +19 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/result.sanitized.json +480 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/transcript.sanitized.jsonl +95 -0
- package/behavioral-evals/evidence-tool.js +443 -0
- package/behavioral-evals/fixtures/claude-denied.jsonl +3 -0
- package/behavioral-evals/fixtures/claude-legacy.json +1 -0
- package/behavioral-evals/fixtures/claude-oral-only.jsonl +2 -0
- package/behavioral-evals/fixtures/claude-success.jsonl +5 -0
- package/behavioral-evals/fixtures/claude-truncated.jsonl +3 -0
- package/behavioral-evals/fixtures/opencode-success.jsonl +6 -0
- package/behavioral-evals/results/.gitkeep +0 -0
- package/behavioral-evals/results/2026-08-23-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-skip-verification.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-sunk-cost-retry.json +20 -0
- package/behavioral-evals/results/2026-08-23-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-23-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-23-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-api-design.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-debugging.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-dependency.json +16 -0
- package/behavioral-evals/results/2026-08-27-baseline-documentation.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-git-commit.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance-monitoring.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor-with-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-security-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-simple-bugfix.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-tdd-workflow.json +24 -0
- package/behavioral-evals/results/2026-08-27-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-budget-skip-review.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-bypass-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-complexity-skip.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-expert-skip-testing.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-fatigue-skip.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-management-skip-all.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-docs.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-security.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-verification.json +24 -0
- package/behavioral-evals/results/2026-08-27-pressure-social-skip-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-sunk-cost-retry.json +24 -0
- package/behavioral-evals/results/2026-08-27-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-27-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-27-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-31-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-09-03-baseline-environment-detection-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-eval-harness-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-fable-discipline-pair.json +149 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-exposure-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-install-cognitive-os-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-multi-agent-workspace-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-rewrite-commits-pair.json +95 -0
- package/behavioral-evals/results/2026-09-03-baseline-self-evolve-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-creator-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-style-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-spec-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-tickets-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-todo-driven-workflow-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-using-git-worktrees-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-summary-both.json +41 -0
- package/behavioral-evals/results/2026-09-04-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-04-baseline-self-evolve-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-baseline-skill-creator-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-summary-both.json +41 -0
- package/behavioral-evals/results/run-20260827-115509.log +287 -0
- package/behavioral-evals/run-with-plugin.js +300 -0
- package/behavioral-evals/run.js +928 -0
- package/behavioral-evals/transcript-parser.js +591 -0
- package/benchmarks/README.md +63 -0
- package/benchmarks/ab-test-2026-08-26/results.json +55 -0
- package/benchmarks/results/logs/.gitkeep +0 -0
- package/benchmarks/results/logs/test-a-harness.md +31 -0
- package/benchmarks/results/logs/test-a-vanilla.md +28 -0
- package/benchmarks/results/logs/test-b-micro-loop-harness.md +48 -0
- package/benchmarks/results/logs/test-b-micro-loop-vanilla.md +36 -0
- package/benchmarks/results/logs/test-c-attention-loss-harness.md +40 -0
- package/benchmarks/results/logs/test-c-attention-loss-vanilla.md +63 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-harness.md +24 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-vanilla.md +29 -0
- package/benchmarks/results/logs/test-e-shell-awareness-harness.md +20 -0
- package/benchmarks/results/logs/test-e-shell-awareness-vanilla.md +20 -0
- package/benchmarks/results/logs/test-f-fact-audit-harness.md +16 -0
- package/benchmarks/results/logs/test-f-fact-audit-vanilla.md +16 -0
- package/benchmarks/results/results.json +158 -0
- package/benchmarks/run.js +220 -0
- package/benchmarks/schema.json +44 -0
- package/bin/cli.js +164 -97
- package/ci/ab-test-cases/grill-me-adversarial.yaml +28 -0
- package/ci/ab-test-cases/tdd-test-first.yaml +30 -0
- package/ci/ab-test-cases/verification-loop-gates.yaml +38 -0
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +19 -0
- package/ci/ab-test-harness.js +485 -0
- package/ci/action-gate-platform-doc.test.js +31 -0
- package/ci/action-gate.test.js +260 -0
- package/{eval-framework → ci}/cases/case1.json +5 -5
- package/ci/cases/case10-small-feature.json +5 -0
- package/ci/cases/case11-readme-typo-update.json +5 -0
- package/ci/cases/case12-chinese-macro-audit.json +5 -0
- package/ci/cases/case2.json +5 -0
- package/{eval-framework → ci}/cases/case3.json +5 -5
- package/{eval-framework → ci}/cases/case4-zh-tier2.json +5 -5
- package/{eval-framework → ci}/cases/case5-zh-tier3.json +5 -5
- package/ci/cases/case6-macro-skill-audit.json +5 -0
- package/ci/cases/case7-macro-reference-audit.json +5 -0
- package/ci/cases/case8-controlled-ab-benchmark.json +5 -0
- package/ci/cases/case9-four-issue-audit.json +5 -0
- package/ci/consistency-check.js +393 -0
- package/ci/description-collision.js +118 -0
- package/ci/disclosure-check.js +225 -0
- package/ci/doc-capability-consistency.test.js +180 -0
- package/ci/fable-mode-test.js +120 -0
- package/ci/fixtures/agency-agents/divisions.json +6 -0
- package/ci/fixtures/agency-agents/engineering/code-reviewer.md +11 -0
- package/ci/fixtures/agency-agents/testing/reality-checker.md +11 -0
- package/ci/fixtures/bad-frontmatter-colon/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-a/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-b/SKILL.md +16 -0
- package/ci/fixtures/phase4/parallel-disjoint.stages.json +32 -0
- package/ci/fixtures/phase4/parallel-read-only.stages.json +32 -0
- package/ci/installer-roundtrip.test.js +159 -0
- package/ci/invariant-routing.test.js +74 -0
- package/ci/issue99-runtime.test.js +218 -0
- package/ci/mechanism-2a-rule-of-3.test.js +46 -0
- package/ci/mechanism-2aa-codex-action-gate-policy.test.js +117 -0
- package/ci/mechanism-2ab-openai-plugin-sync-integrity.test.js +80 -0
- package/ci/mechanism-2b-boundary-guard.test.js +16 -0
- package/ci/mechanism-2c-state-persist.test.js +29 -0
- package/ci/mechanism-2d-fact-audit.test.js +18 -0
- package/ci/mechanism-2e-scope-guard.test.js +24 -0
- package/ci/mechanism-2f-stop-gate.test.js +23 -0
- package/ci/mechanism-2g-platform-ignore.test.js +88 -0
- package/ci/mechanism-2h-installer-manifest.test.js +147 -0
- package/ci/mechanism-2i-crlf-frontmatter.test.js +145 -0
- package/ci/mechanism-2i-install-verification.test.js +34 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +62 -0
- package/ci/mechanism-2k-reference-check.test.js +75 -0
- package/ci/mechanism-2l-multi-agent-workspace.test.js +301 -0
- package/ci/mechanism-2m-self-repo-guard.test.js +65 -0
- package/ci/mechanism-2n-opencode-plugin.test.js +461 -0
- package/ci/mechanism-2o-state-home-isolation.test.js +183 -0
- package/ci/mechanism-2p-installer-links.test.js +183 -0
- package/ci/mechanism-2q-installer-global-shared-store.test.js +132 -0
- package/ci/mechanism-2r-uninstall-flags.test.js +148 -0
- package/ci/mechanism-2s-tdd-quality.test.js +203 -0
- package/ci/mechanism-2t-disclosure.test.js +90 -0
- package/ci/mechanism-2t-evidence-triage.test.js +200 -0
- package/ci/mechanism-2t-issue-20.test.js +75 -0
- package/ci/mechanism-2t-platform-skill-targets.test.js +164 -0
- package/ci/mechanism-2t-transcript-parser.test.js +179 -0
- package/ci/mechanism-2t-workspace-identity.test.js +200 -0
- package/ci/mechanism-2u-issue-42-regressions.test.js +239 -0
- package/ci/mechanism-2u-issue-44.test.js +35 -0
- package/ci/mechanism-2v-verifier-boundaries.test.js +138 -0
- package/ci/mechanism-2w-platform-compatibility.test.js +148 -0
- package/ci/mechanism-2x-release-automation.test.js +157 -0
- package/ci/mechanism-2x-release-commit-message.test.js +66 -0
- package/ci/mechanism-2y-release-tooling-compat.test.js +47 -0
- package/ci/mechanism-2z-claude-plugin-manifest.test.js +29 -0
- package/ci/mechanism-test.js +71 -0
- package/ci/mutation-check.js +53 -0
- package/ci/mutation-loader.js +57 -0
- package/ci/negative-controls.js +114 -0
- package/ci/openai-plugin-package.test.js +182 -0
- package/ci/openai-plugin-runtime.test.js +175 -0
- package/ci/openai-plugin-submission.test.js +101 -0
- package/ci/phase4-ensemble-memory.test.js +307 -0
- package/ci/reference-check.js +160 -0
- package/ci/release-consistency-check.js +146 -0
- package/ci/router-phase2-safety.test.js +81 -0
- package/ci/router-workflow-plan.test.js +264 -0
- package/ci/routing-check.js +91 -0
- package/{eval-framework → ci}/runner.js +57 -56
- package/ci/skill-routing-check.js +135 -0
- package/ci/test-behavioral-case-validation.js +62 -0
- package/ci/test-helper.js +114 -0
- package/ci/test-negative-controls.js +148 -0
- package/ci/workflow-plan-consumers-edge.test.js +72 -0
- package/ci/workflow-plan-consumers.test.js +276 -0
- package/commitlint.config.js +25 -0
- package/docker-verify.sh +266 -266
- package/docs/architecture.md +179 -95
- package/docs/audit.md +55 -0
- package/docs/codex-action-gate.md +90 -0
- package/docs/issue-20-rollup.md +47 -0
- package/docs/mechanism-first-skill-mesh.md +165 -0
- package/docs/openai-plugin.md +118 -0
- package/docs/philosophy.md +58 -45
- package/docs/platform-capabilities.md +73 -0
- package/docs/platform-compatibility.json +270 -0
- package/docs/reflection.md +63 -34
- package/docs/release-evidence.md +25 -0
- package/docs/routing.md +171 -45
- package/docs/security-audit.md +38 -0
- package/docs/troubleshooting.md +320 -0
- package/docs/verifier-boundaries.md +45 -0
- package/docs/workflows/environment-detection.md +10 -10
- package/docs/workflows/eval-harness.md +13 -6
- package/docs/workflows/fable-discipline.md +9 -9
- package/docs/workflows/fable-mode.md +85 -32
- package/docs/workflows/find-skills.md +76 -0
- package/docs/workflows/git-commit.md +18 -9
- package/docs/workflows/grill-me.md +11 -11
- package/docs/workflows/grill-with-docs.md +14 -7
- package/docs/workflows/harness-everything.md +95 -92
- package/docs/workflows/improve-codebase-architecture.md +15 -7
- package/docs/workflows/install-cognitive-os.md +60 -60
- package/docs/workflows/multi-agent-workspace.md +54 -0
- package/docs/workflows/repo-docs.md +12 -7
- package/docs/workflows/security-review.md +26 -7
- package/docs/workflows/self-evolve.md +53 -37
- package/docs/workflows/skill-creator.md +80 -75
- package/docs/workflows/skill-style.md +64 -64
- package/docs/workflows/tdd.md +78 -39
- package/docs/workflows/to-spec.md +83 -0
- package/docs/workflows/to-tickets.md +80 -0
- package/docs/workflows/todo-driven-workflow.md +67 -36
- package/docs/workflows/using-git-worktrees.md +12 -8
- package/docs/workflows/verification-loop.md +16 -14
- package/docs/workflows/verify-before-claim.md +12 -8
- package/docs/workflows/zoom-out.md +22 -10
- package/environment-detection/SKILL.md +25 -61
- package/environment-detection/references/shell-syntax-rules.md +76 -0
- package/environment-detection/scripts/preflight.js +103 -93
- package/eval-framework/fixtures/bad-frontmatter-colon/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-a/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-b/SKILL.md +14 -0
- package/eval-framework/fixtures/missing-delimiter/SKILL.md +13 -0
- package/eval-framework/fixtures/over-budget-skill/SKILL.md +36 -0
- package/eval-harness/SKILL.md +34 -51
- package/eval-harness/references/scoring-rubric.md +68 -0
- package/eval-harness/scripts/evaluate.js +79 -61
- package/evals/environment-detection/eval.yaml +26 -0
- package/evals/environment-detection/tasks/negative-trigger.yaml +16 -0
- package/evals/environment-detection/tasks/positive-trigger.yaml +15 -0
- package/evals/eval-harness/eval.yaml +26 -0
- package/evals/eval-harness/tasks/negative-trigger.yaml +16 -0
- package/evals/eval-harness/tasks/positive-trigger.yaml +15 -0
- package/evals/evaluation-reports.md +14 -0
- package/evals/fable-discipline/eval.yaml +26 -0
- package/evals/fable-discipline/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-discipline/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-mode/eval.yaml +26 -0
- package/evals/fable-mode/tasks/negative-tier2-no-model.yaml +18 -0
- package/evals/fable-mode/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-mode/tasks/positive-haiku-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-opus-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-sonnet-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-trigger.yaml +15 -0
- package/evals/find-skills/eval.yaml +26 -0
- package/evals/find-skills/tasks/negative-trigger.yaml +16 -0
- package/evals/find-skills/tasks/positive-trigger.yaml +15 -0
- package/evals/git-commit/eval.yaml +26 -0
- package/evals/git-commit/tasks/negative-trigger.yaml +16 -0
- package/evals/git-commit/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-me/eval.yaml +26 -0
- package/evals/grill-me/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-me/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-with-docs/eval.yaml +26 -0
- package/evals/grill-with-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-with-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/harness-everything/eval.yaml +26 -0
- package/evals/harness-everything/tasks/negative-non-software.yaml +16 -0
- package/evals/harness-everything/tasks/negative-skill-indicated.yaml +15 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier2.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier3.yaml +16 -0
- package/evals/improve-codebase-architecture/eval.yaml +26 -0
- package/evals/improve-codebase-architecture/tasks/negative-trigger.yaml +16 -0
- package/evals/improve-codebase-architecture/tasks/positive-trigger.yaml +15 -0
- package/evals/install-cognitive-os/eval.yaml +26 -0
- package/evals/install-cognitive-os/tasks/negative-trigger.yaml +16 -0
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/eval.yaml +26 -0
- package/evals/multi-agent-workspace/tasks/negative-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +15 -0
- package/evals/repo-docs/eval.yaml +26 -0
- package/evals/repo-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/repo-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/rewrite-commits/eval.yaml +26 -0
- package/evals/rewrite-commits/tasks/negative-trigger.yaml +16 -0
- package/evals/rewrite-commits/tasks/positive-trigger.yaml +15 -0
- package/evals/security-review/eval.yaml +26 -0
- package/evals/security-review/tasks/negative-trigger.yaml +16 -0
- package/evals/security-review/tasks/positive-trigger.yaml +15 -0
- package/evals/self-evolve/eval.yaml +26 -0
- package/evals/self-evolve/tasks/negative-trigger.yaml +16 -0
- package/evals/self-evolve/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-creator/eval.yaml +26 -0
- package/evals/skill-creator/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-creator/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-style/eval.yaml +26 -0
- package/evals/skill-style/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-style/tasks/positive-trigger.yaml +15 -0
- package/evals/tdd/eval.yaml +26 -0
- package/evals/tdd/tasks/negative-trivia.yaml +16 -0
- package/evals/tdd/tasks/positive-bugfix.yaml +15 -0
- package/evals/tdd/tasks/positive-feature.yaml +16 -0
- package/evals/tdd/tasks/positive-refactor.yaml +15 -0
- package/evals/to-spec/eval.yaml +26 -0
- package/evals/to-spec/tasks/negative-trigger.yaml +16 -0
- package/evals/to-spec/tasks/positive-trigger.yaml +15 -0
- package/evals/to-tickets/eval.yaml +26 -0
- package/evals/to-tickets/tasks/negative-trigger.yaml +15 -0
- package/evals/to-tickets/tasks/positive-trigger.yaml +15 -0
- package/evals/todo-driven-workflow/eval.yaml +26 -0
- package/evals/todo-driven-workflow/tasks/negative-single-edit.yaml +16 -0
- package/evals/todo-driven-workflow/tasks/positive-multi-step.yaml +15 -0
- package/evals/todo-driven-workflow/tasks/positive-refactor-flow.yaml +15 -0
- package/evals/using-git-worktrees/eval.yaml +26 -0
- package/evals/using-git-worktrees/tasks/negative-trigger.yaml +15 -0
- package/evals/using-git-worktrees/tasks/positive-trigger.yaml +15 -0
- package/evals/verification-loop/eval.yaml +26 -0
- package/evals/verification-loop/tasks/negative-discussion.yaml +16 -0
- package/evals/verification-loop/tasks/positive-pre-ship.yaml +15 -0
- package/evals/verify-before-claim/eval.yaml +26 -0
- package/evals/verify-before-claim/tasks/negative-trigger.yaml +16 -0
- package/evals/verify-before-claim/tasks/positive-trigger.yaml +15 -0
- package/evals/zoom-out/eval.yaml +26 -0
- package/evals/zoom-out/tasks/negative-trivial-edit.yaml +16 -0
- package/evals/zoom-out/tasks/positive-repeated-failure.yaml +15 -0
- package/fable-discipline/SKILL.md +31 -28
- package/fable-discipline/references/discipline-rules.md +22 -0
- package/fable-mode/CONTRACT-FORMAT.md +165 -51
- package/fable-mode/SKILL.md +31 -43
- package/fable-mode/agents/fable-orchestrator.md +159 -92
- package/fable-mode/agents/fable-verifier.md +32 -32
- package/fable-mode/agents/fable-worker-haiku.md +52 -48
- package/fable-mode/agents/fable-worker-sonnet.md +53 -49
- package/fable-mode/execution-guardrails/SKILL.md +43 -87
- package/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/fable-mode/fable-haiku/SKILL.md +36 -48
- package/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/fable-mode/fable-opus/SKILL.md +36 -0
- package/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/fable-mode/model-matrix.json +32 -0
- package/fable-mode/references/execution-phases.md +37 -0
- package/fable-mode/references/model-matrix.md +39 -0
- package/fable-mode/scripts/ensemble-review.js +371 -0
- package/fable-mode/scripts/model-selector.js +260 -0
- package/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/find-skills/SKILL.md +48 -0
- package/find-skills/references/discovery-flow.md +102 -0
- package/find-skills/scripts/use-skill.js +112 -0
- package/git-commit/SKILL.md +28 -45
- package/git-commit/guides/ANGULAR_STYLE.md +60 -60
- package/git-commit/guides/COMMIT_GENERATION.md +23 -23
- package/git-commit/guides/LANGUAGE_DETECTION.md +16 -16
- package/git-commit/guides/MAIN_REPO.md +14 -14
- package/git-commit/guides/SUBMODULES.md +27 -27
- package/git-commit/references/commit-flow.md +30 -0
- package/grill-me/SKILL.md +34 -27
- package/grill-me/references/grilling-playbook.md +38 -0
- package/grill-with-docs/ADR-FORMAT.md +47 -47
- package/grill-with-docs/CONTEXT-FORMAT.md +63 -63
- package/grill-with-docs/SKILL.md +33 -91
- package/grill-with-docs/references/session-playbook.md +99 -0
- package/harness-everything/SKILL.md +27 -125
- package/harness-everything/examples/README.md +9 -9
- package/harness-everything/examples/todo-workflow.md +22 -22
- package/harness-everything/references/router-workflow-plan.md +128 -0
- package/harness-everything/references/skill-registry.md +39 -0
- package/harness-everything/references/triage-and-tiers.md +193 -0
- package/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/harness-everything/scripts/bootstrap.js +212 -140
- package/harness-everything/scripts/ensemble-policy.js +102 -0
- package/harness-everything/scripts/kernel-router.js +212 -0
- package/harness-everything/scripts/router-contract.js +612 -0
- package/harness-everything/scripts/routing-keywords.json +320 -0
- package/harness-everything/scripts/self-heal.js +250 -189
- package/harness-everything/scripts/tier-router.js +459 -219
- package/harness-everything/scripts/verify-gate.js +100 -31
- package/hooks/hooks.json +200 -155
- package/hooks/scripts/action-gate-rules.json +77 -0
- package/hooks/scripts/action-gate.js +528 -0
- package/hooks/scripts/atomic-commit-check.js +68 -68
- package/hooks/scripts/boundary-guard.js +55 -55
- package/hooks/scripts/codex-action-gate-post.js +46 -0
- package/hooks/scripts/codex-permission-request.js +140 -0
- package/hooks/scripts/context-compact.js +115 -114
- package/hooks/scripts/contract-test.js +146 -79
- package/hooks/scripts/depth-guard.js +65 -65
- package/hooks/scripts/lib/fable-contracts.js +121 -0
- package/hooks/scripts/lib/harness-state.js +275 -228
- package/hooks/scripts/lib/platforms/claude.js +97 -61
- package/hooks/scripts/lib/platforms/codex.js +42 -50
- package/hooks/scripts/lib/platforms/continue.js +30 -12
- package/hooks/scripts/lib/platforms/copilot.js +36 -42
- package/hooks/scripts/lib/platforms/cursor.js +23 -5
- package/hooks/scripts/lib/platforms/hermes.js +108 -57
- package/hooks/scripts/lib/platforms/index.js +17 -17
- package/hooks/scripts/lib/platforms/worktrees.js +20 -20
- package/hooks/scripts/openai-session-start.js +81 -0
- package/hooks/scripts/reset-circuit-breaker.js +30 -30
- package/hooks/scripts/rule-of-3-tracker.js +125 -91
- package/hooks/scripts/rule-of-3.js +195 -176
- package/hooks/scripts/state-persist.js +100 -99
- package/hooks/scripts/stop-gate.js +80 -80
- package/hooks/scripts/subagent-scope-guard.js +194 -88
- package/hooks/scripts/subagent-start.js +19 -0
- package/hooks/scripts/subagent-stop.js +6 -0
- package/improve-codebase-architecture/SKILL.md +29 -35
- package/improve-codebase-architecture/guides/DEEPENING.md +37 -37
- package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -123
- package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -44
- package/improve-codebase-architecture/guides/LANGUAGE.md +53 -53
- package/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/improve-codebase-architecture/references/README.md +16 -0
- package/install-cognitive-os/SKILL.md +30 -81
- package/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/multi-agent-workspace/SKILL.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/examples/csharp-monorepo.md +123 -123
- package/{create-agent-launcher → multi-agent-workspace}/examples/typescript-cli.md +126 -126
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-claude.md +77 -77
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-codex.md +146 -146
- package/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-copilot.md +74 -74
- package/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/role-capabilities.md +108 -108
- package/multi-agent-workspace/references/agency-agents.md +31 -0
- package/multi-agent-workspace/references/architecture-guide.md +36 -0
- package/multi-agent-workspace/references/orchestration.md +65 -0
- package/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/multi-agent-workspace/scripts/index_memory.js +104 -0
- package/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/multi-agent-workspace/scripts/scaffold.js +408 -0
- package/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/{create-agent-launcher → multi-agent-workspace}/templates/README.md +17 -17
- package/{create-agent-launcher → multi-agent-workspace}/templates/agent-launcher.md +67 -67
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-csharp.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-polyglot.md +61 -61
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-ts.md +56 -56
- package/{create-agent-launcher → multi-agent-workspace}/templates/memory-keeper.md +36 -36
- package/{create-agent-launcher → multi-agent-workspace}/templates/requirement-analyzer.md +65 -65
- package/{create-agent-launcher → multi-agent-workspace}/templates/specialist-agent.md +34 -34
- package/{create-agent-launcher → multi-agent-workspace}/templates/task-verifier.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/tool-maker.md +32 -32
- package/multi-agent-workspace/workflows/01-init.md +22 -0
- package/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/multi-agent-workspace/workflows/03-generation.md +25 -0
- package/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/opencode-plugin/README.md +137 -0
- package/opencode-plugin/hooks/verify.js +190 -0
- package/opencode-plugin/index.mjs +547 -0
- package/opencode-plugin/plugin.json +9 -0
- package/package.json +63 -22
- package/plugins/harness-everything/.codex-plugin/plugin.json +39 -0
- package/plugins/harness-everything/hooks/hooks.json +197 -0
- package/plugins/harness-everything/hooks/scripts/action-gate-rules.json +77 -0
- package/plugins/harness-everything/hooks/scripts/action-gate.js +528 -0
- package/plugins/harness-everything/hooks/scripts/atomic-commit-check.js +68 -0
- package/plugins/harness-everything/hooks/scripts/boundary-guard.js +55 -0
- package/plugins/harness-everything/hooks/scripts/codex-action-gate-post.js +46 -0
- package/plugins/harness-everything/hooks/scripts/codex-permission-request.js +140 -0
- package/plugins/harness-everything/hooks/scripts/context-compact.js +115 -0
- package/plugins/harness-everything/hooks/scripts/contract-test.js +146 -0
- package/plugins/harness-everything/hooks/scripts/depth-guard.js +65 -0
- package/plugins/harness-everything/hooks/scripts/lib/fable-contracts.js +121 -0
- package/plugins/harness-everything/hooks/scripts/lib/harness-state.js +275 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/claude.js +168 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/codex.js +88 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/continue.js +91 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/copilot.js +85 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/cursor.js +89 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/hermes.js +108 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/index.js +17 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/worktrees.js +20 -0
- package/plugins/harness-everything/hooks/scripts/lib/workspace.js +290 -0
- package/plugins/harness-everything/hooks/scripts/openai-session-start.js +81 -0
- package/plugins/harness-everything/hooks/scripts/reset-circuit-breaker.js +30 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3-tracker.js +125 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3.js +195 -0
- package/plugins/harness-everything/hooks/scripts/state-persist.js +100 -0
- package/plugins/harness-everything/hooks/scripts/stop-gate.js +80 -0
- package/plugins/harness-everything/hooks/scripts/subagent-scope-guard.js +194 -0
- package/plugins/harness-everything/hooks/scripts/subagent-start.js +19 -0
- package/plugins/harness-everything/hooks/scripts/subagent-stop.js +6 -0
- package/plugins/harness-everything/hooks/session-start.js +81 -0
- package/plugins/harness-everything/plugin.json +43 -0
- package/plugins/harness-everything/scripts/lib/advisory-text.js +174 -0
- package/plugins/harness-everything/skills/environment-detection/SKILL.md +41 -0
- package/plugins/harness-everything/skills/environment-detection/references/shell-syntax-rules.md +76 -0
- package/plugins/harness-everything/skills/environment-detection/scripts/preflight.js +103 -0
- package/plugins/harness-everything/skills/eval-harness/SKILL.md +42 -0
- package/plugins/harness-everything/skills/eval-harness/references/scoring-rubric.md +68 -0
- package/plugins/harness-everything/skills/eval-harness/scripts/evaluate.js +79 -0
- package/plugins/harness-everything/skills/fable-discipline/SKILL.md +41 -0
- package/plugins/harness-everything/skills/fable-discipline/references/discipline-rules.md +22 -0
- package/plugins/harness-everything/skills/fable-mode/CONTRACT-FORMAT.md +165 -0
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +159 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-verifier.md +32 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-haiku.md +52 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-sonnet.md +53 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +43 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/model-matrix.json +32 -0
- package/plugins/harness-everything/skills/fable-mode/references/execution-phases.md +37 -0
- package/plugins/harness-everything/skills/fable-mode/references/model-matrix.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/ensemble-review.js +371 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/model-selector.js +260 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/plugins/harness-everything/skills/find-skills/SKILL.md +48 -0
- package/plugins/harness-everything/skills/find-skills/references/discovery-flow.md +102 -0
- package/plugins/harness-everything/skills/find-skills/scripts/use-skill.js +112 -0
- package/plugins/harness-everything/skills/git-commit/SKILL.md +38 -0
- package/plugins/harness-everything/skills/git-commit/guides/ANGULAR_STYLE.md +61 -0
- package/plugins/harness-everything/skills/git-commit/guides/COMMIT_GENERATION.md +24 -0
- package/plugins/harness-everything/skills/git-commit/guides/LANGUAGE_DETECTION.md +16 -0
- package/plugins/harness-everything/skills/git-commit/guides/MAIN_REPO.md +15 -0
- package/plugins/harness-everything/skills/git-commit/guides/SUBMODULES.md +27 -0
- package/plugins/harness-everything/skills/git-commit/references/commit-flow.md +30 -0
- package/plugins/harness-everything/skills/grill-me/SKILL.md +43 -0
- package/plugins/harness-everything/skills/grill-me/references/grilling-playbook.md +38 -0
- package/plugins/harness-everything/skills/grill-with-docs/ADR-FORMAT.md +47 -0
- package/plugins/harness-everything/skills/grill-with-docs/CONTEXT-FORMAT.md +63 -0
- package/plugins/harness-everything/skills/grill-with-docs/SKILL.md +41 -0
- package/plugins/harness-everything/skills/grill-with-docs/references/session-playbook.md +99 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +43 -0
- package/plugins/harness-everything/skills/harness-everything/examples/README.md +9 -0
- package/plugins/harness-everything/skills/harness-everything/examples/todo-workflow.md +22 -0
- package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md +128 -0
- package/plugins/harness-everything/skills/harness-everything/references/skill-registry.md +39 -0
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +193 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/bootstrap.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/ensemble-policy.js +102 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/router-contract.js +612 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/routing-keywords.json +320 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/self-heal.js +250 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +459 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/verify-gate.js +100 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/SKILL.md +38 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/DEEPENING.md +37 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/HTML-REPORT.md +123 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/LANGUAGE.md +53 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/references/README.md +16 -0
- package/plugins/harness-everything/skills/install-cognitive-os/SKILL.md +44 -0
- package/plugins/harness-everything/skills/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/csharp-monorepo.md +123 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/typescript-cli.md +126 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-claude.md +77 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-codex.md +146 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-copilot.md +74 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/role-capabilities.md +108 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/agency-agents.md +31 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/architecture-guide.md +36 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/orchestration.md +65 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/index_memory.js +104 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/scaffold.js +408 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/README.md +17 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/agent-launcher.md +68 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-csharp.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-polyglot.md +62 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-ts.md +57 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/memory-keeper.md +37 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/requirement-analyzer.md +66 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/specialist-agent.md +34 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/task-verifier.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/tool-maker.md +33 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/01-init.md +22 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/03-generation.md +25 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/plugins/harness-everything/skills/repo-docs/SKILL.md +42 -0
- package/plugins/harness-everything/skills/repo-docs/references/process-guide.md +60 -0
- package/plugins/harness-everything/skills/repo-docs/templates/agents-template.md +264 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-agents-template.md +50 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-readme-template.md +83 -0
- package/plugins/harness-everything/skills/repo-docs/templates/manual-recon.md +117 -0
- package/plugins/harness-everything/skills/repo-docs/templates/multi-skills-readme-template.md +72 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-agents-template.md +78 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-readme-template.md +95 -0
- package/plugins/harness-everything/skills/repo-docs/templates/readme-template.md +136 -0
- package/plugins/harness-everything/skills/repo-docs/templates/single-skill-readme-template.md +70 -0
- package/plugins/harness-everything/skills/repo-docs/templates/skills-agents-template.md +60 -0
- package/plugins/harness-everything/skills/rewrite-commits/SKILL.md +37 -0
- package/plugins/harness-everything/skills/rewrite-commits/references/workflow.md +18 -0
- package/plugins/harness-everything/skills/security-review/SKILL.md +47 -0
- package/plugins/harness-everything/skills/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/plugins/harness-everything/skills/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/plugins/harness-everything/skills/security-review/references/security-checklist.md +478 -0
- package/plugins/harness-everything/skills/security-review/scripts/audit-secrets.js +85 -0
- package/plugins/harness-everything/skills/self-evolve/SKILL.md +45 -0
- package/plugins/harness-everything/skills/self-evolve/references/memory-resolution.md +61 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/persist-memory.js +272 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/register-dynamic-skill.js +253 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/self-regression.js +125 -0
- package/plugins/harness-everything/skills/skill-creator/SKILL.md +47 -0
- package/plugins/harness-everything/skills/skill-creator/references/authoring-workflow.md +14 -0
- package/plugins/harness-everything/skills/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-checklist.md +16 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-principles.md +81 -0
- package/plugins/harness-everything/skills/skill-creator/references/testing-workflow.md +27 -0
- package/plugins/harness-everything/skills/skill-style/SKILL.md +42 -0
- package/plugins/harness-everything/skills/skill-style/references/style-guide.md +62 -0
- package/plugins/harness-everything/skills/tdd/SKILL.md +38 -0
- package/plugins/harness-everything/skills/tdd/fixtures/pass-evidence.json +100 -0
- package/plugins/harness-everything/skills/tdd/fixtures/quality-scenarios.json +20 -0
- package/plugins/harness-everything/skills/tdd/guides/deep-modules.md +33 -0
- package/plugins/harness-everything/skills/tdd/guides/interface-design.md +31 -0
- package/plugins/harness-everything/skills/tdd/guides/mocking.md +59 -0
- package/plugins/harness-everything/skills/tdd/guides/refactoring.md +10 -0
- package/plugins/harness-everything/skills/tdd/guides/tests.md +61 -0
- package/plugins/harness-everything/skills/tdd/references/core-discipline.md +51 -0
- package/plugins/harness-everything/skills/tdd/references/integration-testing.md +13 -0
- package/plugins/harness-everything/skills/tdd/references/quality-model.md +68 -0
- package/plugins/harness-everything/skills/tdd/references/unit-testing.md +15 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-evidence.schema.json +211 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-report.schema.json +110 -0
- package/plugins/harness-everything/skills/tdd/scripts/quality-gate.js +475 -0
- package/plugins/harness-everything/skills/to-spec/SKILL.md +40 -0
- package/plugins/harness-everything/skills/to-spec/references/process.md +77 -0
- package/plugins/harness-everything/skills/to-spec/scripts/check-project-docs.js +239 -0
- package/plugins/harness-everything/skills/to-spec/scripts/project-docs-resolver.js +111 -0
- package/plugins/harness-everything/skills/to-spec/templates/cli-reference.md +31 -0
- package/plugins/harness-everything/skills/to-spec/templates/dev-doc.md +27 -0
- package/plugins/harness-everything/skills/to-spec/templates/feature-spec.md +59 -0
- package/plugins/harness-everything/skills/to-spec/templates/schema-doc.md +31 -0
- package/plugins/harness-everything/skills/to-tickets/SKILL.md +42 -0
- package/plugins/harness-everything/skills/to-tickets/references/publishing-and-templates.md +97 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/SKILL.md +42 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/references/execution-guide.md +27 -0
- package/plugins/harness-everything/skills/using-git-worktrees/SKILL.md +49 -0
- package/plugins/harness-everything/skills/using-git-worktrees/references/workflow-details.md +108 -0
- package/plugins/harness-everything/skills/verification-loop/SKILL.md +46 -0
- package/plugins/harness-everything/skills/verification-loop/references/verification-phases.md +116 -0
- package/plugins/harness-everything/skills/verification-loop/templates/verification-report.template.md +15 -0
- package/plugins/harness-everything/skills/verify-before-claim/SKILL.md +43 -0
- package/plugins/harness-everything/skills/verify-before-claim/references/verification-guide.md +73 -0
- package/plugins/harness-everything/skills/zoom-out/SKILL.md +42 -0
- package/plugins/harness-everything/skills/zoom-out/references/circuit-breaker.md +66 -0
- package/plugins/harness-everything/skills/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/references/definition-of-done.md +67 -0
- package/references/performance-checklist.md +153 -0
- package/references/platform-advisory-template.md +58 -0
- package/references/security-checklist.md +205 -0
- package/repo-docs/SKILL.md +34 -37
- package/repo-docs/references/process-guide.md +60 -0
- package/repo-docs/templates/agents-template.md +264 -264
- package/repo-docs/templates/knowledge-agents-template.md +50 -50
- package/repo-docs/templates/knowledge-readme-template.md +83 -83
- package/repo-docs/templates/manual-recon.md +117 -117
- package/repo-docs/templates/multi-skills-readme-template.md +72 -72
- package/repo-docs/templates/product-agents-template.md +78 -78
- package/repo-docs/templates/product-readme-template.md +95 -95
- package/repo-docs/templates/readme-template.md +136 -136
- package/repo-docs/templates/single-skill-readme-template.md +70 -70
- package/repo-docs/templates/skills-agents-template.md +60 -60
- package/rewrite-commits/SKILL.md +22 -17
- package/rewrite-commits/references/workflow.md +18 -0
- package/scripts/build-openai-submission.js +158 -0
- package/scripts/installer.js +621 -506
- package/scripts/lib/advisory-text.js +33 -22
- package/scripts/lib/claude-hooks.js +18 -5
- package/scripts/lib/execution-contract.js +70 -0
- package/scripts/lib/gitignore.js +199 -80
- package/scripts/lib/manifest.js +83 -23
- package/scripts/lib/path-boundary.js +90 -0
- package/scripts/lib/prompts.js +1 -1
- package/scripts/lib/skills.js +191 -51
- package/scripts/lib/workspace.js +269 -7
- package/scripts/sync-openai-plugin.js +91 -0
- package/scripts/sync-release-version.js +212 -0
- package/scripts/verify-install.js +147 -0
- package/security-review/SKILL.md +33 -497
- package/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/security-review/references/security-checklist.md +478 -0
- package/security-review/scripts/audit-secrets.js +85 -0
- package/self-evolve/SKILL.md +36 -45
- package/self-evolve/references/memory-resolution.md +61 -0
- package/self-evolve/scripts/persist-memory.js +272 -40
- package/self-evolve/scripts/register-dynamic-skill.js +253 -0
- package/self-evolve/scripts/self-regression.js +125 -113
- package/skill-creator/SKILL.md +33 -77
- package/skill-creator/references/authoring-workflow.md +14 -0
- package/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/skill-creator/references/quality-checklist.md +16 -0
- package/skill-creator/references/quality-principles.md +81 -81
- package/skill-creator/references/testing-workflow.md +27 -32
- package/skill-style/SKILL.md +33 -39
- package/skill-style/references/style-guide.md +62 -0
- package/submission/openai/README.md +67 -0
- package/submission/openai/listing.json +31 -0
- package/submission/openai/test-cases.json +61 -0
- package/tdd/SKILL.md +30 -45
- package/tdd/fixtures/pass-evidence.json +100 -0
- package/tdd/fixtures/quality-scenarios.json +20 -0
- package/tdd/guides/deep-modules.md +33 -33
- package/tdd/guides/interface-design.md +31 -31
- package/tdd/guides/mocking.md +59 -59
- package/tdd/guides/refactoring.md +10 -10
- package/tdd/guides/tests.md +61 -61
- package/tdd/references/core-discipline.md +51 -0
- package/tdd/references/integration-testing.md +13 -0
- package/tdd/references/quality-model.md +68 -0
- package/tdd/references/unit-testing.md +15 -0
- package/tdd/schemas/quality-evidence.schema.json +211 -0
- package/tdd/schemas/quality-report.schema.json +110 -0
- package/tdd/scripts/quality-gate.js +475 -0
- package/to-spec/SKILL.md +40 -0
- package/to-spec/references/process.md +77 -0
- package/to-spec/scripts/check-project-docs.js +239 -0
- package/to-spec/scripts/project-docs-resolver.js +111 -0
- package/to-spec/templates/cli-reference.md +31 -0
- package/to-spec/templates/dev-doc.md +27 -0
- package/to-spec/templates/feature-spec.md +59 -0
- package/to-spec/templates/schema-doc.md +31 -0
- package/to-tickets/SKILL.md +42 -0
- package/to-tickets/references/publishing-and-templates.md +97 -0
- package/todo-driven-workflow/SKILL.md +28 -40
- package/todo-driven-workflow/references/execution-guide.md +27 -0
- package/using-git-worktrees/SKILL.md +29 -141
- package/using-git-worktrees/references/workflow-details.md +108 -0
- package/verification-loop/SKILL.md +29 -121
- package/verification-loop/references/verification-phases.md +116 -0
- package/verification-loop/templates/verification-report.template.md +15 -0
- package/verify-before-claim/SKILL.md +27 -70
- package/verify-before-claim/references/verification-guide.md +73 -0
- package/zoom-out/SKILL.md +28 -76
- package/zoom-out/references/circuit-breaker.md +66 -0
- package/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/build-multi-agent-system/AGENTS.md +0 -50
- package/build-multi-agent-system/README.md +0 -63
- package/build-multi-agent-system/SKILL.md +0 -68
- package/create-agent-launcher/SKILL.md +0 -48
- package/create-agent-launcher/guidelines/platform-gemini.md +0 -101
- package/create-agent-launcher/workflows/01-init.md +0 -22
- package/create-agent-launcher/workflows/02-analysis.md +0 -26
- package/create-agent-launcher/workflows/03-generation.md +0 -29
- package/create-agent-launcher/workflows/04-launcher.md +0 -20
- package/docs/workflows/build-multi-agent-system.md +0 -62
- package/docs/workflows/create-agent-launcher.md +0 -62
- package/eval-framework/behavioral-test.js +0 -105
- package/eval-framework/cases/case2.json +0 -5
- package/eval-framework/mechanism-test.js +0 -251
- package/harness-everything/scripts/todo-cli.js +0 -157
- package/skill-style/STYLE.md +0 -37
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
const { execSync } = require('child_process');
|
|
3
|
+
const os = require('os');
|
|
4
|
+
const fs = require('fs');
|
|
5
|
+
const path = require('path');
|
|
6
|
+
|
|
7
|
+
function checkCmd(cmd) {
|
|
8
|
+
try {
|
|
9
|
+
if (os.platform() === 'win32') {
|
|
10
|
+
execSync(`where "${cmd}"`, { stdio: 'ignore' });
|
|
11
|
+
} else {
|
|
12
|
+
execSync(`which "${cmd}"`, { stdio: 'ignore' });
|
|
13
|
+
}
|
|
14
|
+
return true;
|
|
15
|
+
} catch (e) {
|
|
16
|
+
try {
|
|
17
|
+
execSync(`${cmd} --version`, { stdio: 'ignore' });
|
|
18
|
+
return true;
|
|
19
|
+
} catch (e2) {
|
|
20
|
+
return false;
|
|
21
|
+
}
|
|
22
|
+
}
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
console.log("# [Environment Preflight Report]");
|
|
26
|
+
console.log(`- **Operating System**: ${os.platform()} (${os.release()} ${os.arch()})`);
|
|
27
|
+
|
|
28
|
+
// Shell Detection
|
|
29
|
+
const isGitBash = !!(
|
|
30
|
+
process.env.MSYSTEM ||
|
|
31
|
+
process.env.BASH_VERSION ||
|
|
32
|
+
(process.env.SHELL && process.env.SHELL.toLowerCase().includes("bash"))
|
|
33
|
+
);
|
|
34
|
+
const isPowerShell = !!(
|
|
35
|
+
process.env.PSModulePath ||
|
|
36
|
+
(process.env.SHELL && (process.env.SHELL.toLowerCase().includes("powershell") || process.env.SHELL.toLowerCase().includes("pwsh")))
|
|
37
|
+
);
|
|
38
|
+
|
|
39
|
+
let shell = process.env.SHELL || (isGitBash ? "bash" : (isPowerShell ? "powershell" : process.env.COMSPEC)) || "Unknown";
|
|
40
|
+
let shellName = "Unknown Shell";
|
|
41
|
+
|
|
42
|
+
if (isGitBash || shell.toLowerCase().includes("bash")) {
|
|
43
|
+
shellName = "Git Bash / Bash";
|
|
44
|
+
} else if (isPowerShell) {
|
|
45
|
+
shellName = "PowerShell";
|
|
46
|
+
} else if (shell.toLowerCase().includes("cmd.exe")) {
|
|
47
|
+
shellName = "Windows Command Prompt (CMD)";
|
|
48
|
+
} else {
|
|
49
|
+
shellName = shell;
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
console.log(`- **Active Shell**: ${shellName} (Path: \`${shell}\`)`);
|
|
53
|
+
console.log(`- **Terminal Program**: ${process.env.TERM_PROGRAM || "Standard Terminal"}`);
|
|
54
|
+
|
|
55
|
+
// CLI Tools Check
|
|
56
|
+
const tools = ['git', 'docker', 'docker-compose', 'node', 'pnpm', 'npm', 'python', 'pip'];
|
|
57
|
+
const availableTools = [];
|
|
58
|
+
for (const tool of tools) {
|
|
59
|
+
if (checkCmd(tool)) {
|
|
60
|
+
availableTools.push(tool);
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
console.log(`- **Available CLI Tools**: ${availableTools.join(', ') || 'None'}`);
|
|
64
|
+
|
|
65
|
+
// Sandbox Capabilities Detection
|
|
66
|
+
const hasDocker = availableTools.includes('docker');
|
|
67
|
+
const hasSandboxExec = checkCmd('sandbox-exec');
|
|
68
|
+
const sandboxCapabilities = [];
|
|
69
|
+
if (hasSandboxExec) sandboxCapabilities.push('macOS Seatbelt (sandbox-exec)');
|
|
70
|
+
if (hasDocker) sandboxCapabilities.push('Docker Containers');
|
|
71
|
+
if (os.platform() === 'win32') {
|
|
72
|
+
sandboxCapabilities.push('Windows AppContainer (Native)');
|
|
73
|
+
}
|
|
74
|
+
console.log(`- **Sandbox Capabilities**: ${sandboxCapabilities.join(', ') || 'No isolation sandbox detected'}`);
|
|
75
|
+
|
|
76
|
+
// Repo Status
|
|
77
|
+
try {
|
|
78
|
+
const branch = execSync('git branch --show-current', { encoding: 'utf8' }).trim();
|
|
79
|
+
console.log(`- **Git Workspace**: Detected (Branch: \`${branch}\`)`);
|
|
80
|
+
} catch (e) {
|
|
81
|
+
console.log(`- **Git Workspace**: No git repository detected in ${process.cwd()}`);
|
|
82
|
+
}
|
|
83
|
+
|
|
84
|
+
// Recommended Syntax Guardrails
|
|
85
|
+
console.log("\n## [Syntax & Command Guardrails for this Environment]");
|
|
86
|
+
if (os.platform() === 'win32') {
|
|
87
|
+
if (shellName.includes("Bash")) {
|
|
88
|
+
console.log("- **Path Handling**: ALWAYS use `/` slashes for directory paths in commands (e.g., `scripts/run.js`). Avoid raw backslashes `\\` as they act as escape chars.");
|
|
89
|
+
console.log("- **Environment Variables**: Use `$VAR` syntax. Do NOT use `%VAR%` or `$env:VAR`.");
|
|
90
|
+
console.log("- **Command Chaining**: Use `&&` for sequence, `|` for piping.");
|
|
91
|
+
console.log("- **Avoid Native Windows Commands**: Do NOT use `dir`, `copy`, or `del`. Use bash standard equivalents `ls`, `cp`, `rm`.");
|
|
92
|
+
} else if (shellName.includes("PowerShell")) {
|
|
93
|
+
console.log("- **Path Handling**: Slashes `/` or `\\` are generally acceptable.");
|
|
94
|
+
console.log("- **Environment Variables**: Use `$env:VAR` syntax.");
|
|
95
|
+
console.log("- **Command Chaining**: Use `;` or `&&` (if PowerShell 7+). Avoid bash-specific commands like `grep`, `awk`, `sed` unless explicitly piping.");
|
|
96
|
+
} else {
|
|
97
|
+
console.log("- **Path Handling**: Use `\\` for directories.");
|
|
98
|
+
console.log("- **Environment Variables**: Use `%VAR%`.");
|
|
99
|
+
console.log("- **Chaining**: Use `&`.");
|
|
100
|
+
}
|
|
101
|
+
} else {
|
|
102
|
+
console.log("- **Unix Native**: Standard Bash/Zsh syntax applies. Use relative paths, pipeline chaining, and standard utilities.");
|
|
103
|
+
}
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: eval-harness
|
|
3
|
+
description: Evaluate AI agent performance across correctness, token efficiency, anti-loop focus, and environment awareness; save a 4-dimension scorecard. Use when asked to run a benchmark or score an execution log or conversation history.
|
|
4
|
+
license: Apache-2.0
|
|
5
|
+
metadata:
|
|
6
|
+
author: Miya Daniel
|
|
7
|
+
version: 0.4.0
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Eval Harness
|
|
11
|
+
|
|
12
|
+
Evaluate an agent log with a four-dimension scorecard.
|
|
13
|
+
|
|
14
|
+
## Skill Contract
|
|
15
|
+
|
|
16
|
+
| Component | Specification |
|
|
17
|
+
| :--- | :--- |
|
|
18
|
+
| **Trigger / Input** | A benchmark request and an execution log or conversation history. |
|
|
19
|
+
| **Expected Output** | A 0–10 scorecard for correctness, efficiency, anti-loop focus, and environment awareness. |
|
|
20
|
+
| **State Mutations** | Writes the scorecard under `<workspace>/evals/` or `<workspace>/.github/harness-everything/evals/`. |
|
|
21
|
+
| **Enforcement Gate** | Run `<this-skill-dir>/scripts/evaluate.js`; use the documented fallback only if it fails or is missing. |
|
|
22
|
+
|
|
23
|
+
## Workflow
|
|
24
|
+
|
|
25
|
+
1. Parse actions, errors, tokens, and timing from the log.
|
|
26
|
+
2. Score all four dimensions using `<this-skill-dir>/references/scoring-rubric.md`.
|
|
27
|
+
3. Run `node <this-skill-dir>/scripts/evaluate.js <A> <B> <C> <D> "insights"`.
|
|
28
|
+
4. If the script fails, write `<workspace>/evals/scorecard.md` (or the `<workspace>/.github` fallback).
|
|
29
|
+
|
|
30
|
+
## USE FOR:
|
|
31
|
+
|
|
32
|
+
- Running a benchmark or scoring an agent execution.
|
|
33
|
+
- Comparing Vanilla and Harness execution.
|
|
34
|
+
- Generating a four-dimension scorecard.
|
|
35
|
+
|
|
36
|
+
## DO NOT USE FOR:
|
|
37
|
+
|
|
38
|
+
- Writing or fixing the evaluated code.
|
|
39
|
+
- General review without scoring dimensions.
|
|
40
|
+
- Tasks without an execution log.
|
|
41
|
+
|
|
42
|
+
Deep dive: `<this-skill-dir>/references/scoring-rubric.md`
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# Eval Harness — Full Process & Scoring Rubric
|
|
2
|
+
|
|
3
|
+
## Process & Evaluation Output Flow
|
|
4
|
+
|
|
5
|
+
Follow the decision matrix below to evaluate performance and record scorecards:
|
|
6
|
+
|
|
7
|
+
```mermaid
|
|
8
|
+
flowchart TD
|
|
9
|
+
Start[Trigger: Benchmark / Evaluate Log] --> Parse[1. Parse Log: Actions, Loops, Tokens]
|
|
10
|
+
Parse --> Score[2. Calculate 4-Dimension Rubric Scores]
|
|
11
|
+
|
|
12
|
+
Score --> CheckScript{3. Is evaluate.js Executable?}
|
|
13
|
+
|
|
14
|
+
CheckScript -- Yes --> RunScript[Run node "<this-skill-dir>/scripts/evaluate.js" <scores> <insights>]
|
|
15
|
+
RunScript -- Exit 0 (Success) --> Done[Scorecard Saved to evals/]
|
|
16
|
+
RunScript -- Fails --> FileFallback
|
|
17
|
+
|
|
18
|
+
CheckScript -- No --> FileFallback{4. Markdown File Fallback}
|
|
19
|
+
|
|
20
|
+
FileFallback -- evals/ Directory Exists --> WriteEvals[Write to evals/scorecard.md]
|
|
21
|
+
FileFallback -- No evals/ Directory --> WritePlatform[Write to .github/harness-everything/evals/scorecard.md]
|
|
22
|
+
|
|
23
|
+
WriteEvals --> Done
|
|
24
|
+
WritePlatform --> Done
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## 1. Data Parsing `[Discover]`
|
|
28
|
+
- Read the execution logs provided by the user or directly analyze the current conversation history.
|
|
29
|
+
- Extract key metrics:
|
|
30
|
+
- **Action Count**: Includes the number of file reads, searches, and terminal executions.
|
|
31
|
+
- **Error Loop Count**: The number of times the same error message appeared consecutively triggering attempted micro-adjustments.
|
|
32
|
+
- **Token / Time Indicator**: Record token consumption or time spent if provided by the user.
|
|
33
|
+
|
|
34
|
+
## 2. Scoring Rubric `[Think]`
|
|
35
|
+
|
|
36
|
+
Score based on the following four dimensions (out of 10 points each):
|
|
37
|
+
|
|
38
|
+
### A. Correctness & Factuality
|
|
39
|
+
- 10 points: Perfectly solved the problem with no side effects. Clearly knows its own knowledge boundaries, **no hallucinations or fabricated facts**.
|
|
40
|
+
- 5 points: Solved the surface problem but introduced new issues, or provided slightly outdated information.
|
|
41
|
+
- 0 points: Task failed, broke original code, or "hallucinated with straight-faced confidence".
|
|
42
|
+
|
|
43
|
+
### B. Token & Step Efficiency
|
|
44
|
+
- 10 points: Extremely streamlined, no superfluous exploration or useless tool calls (prevented over-engineering).
|
|
45
|
+
- 5 points: Reasonable steps, but with minor unnecessary file reads.
|
|
46
|
+
- 0 points: Over-engineered, writing lengthy plans or repeatedly reading irrelevant files for a simple task.
|
|
47
|
+
|
|
48
|
+
### C. Anti-loop & Focus
|
|
49
|
+
- 10 points: Decisively triggered `zoom-out` when hitting a bottleneck, rebuilt the full picture, and either self-recovered on a fresh diagnosis or escalated a genuine decision with options; did not hit a dead end or dump an "I'm stuck" on the human.
|
|
50
|
+
- 5 points: Stopped after 3~5 attempts.
|
|
51
|
+
- 0 points: Fell into a Micro-error looping cycle more than 5 times, or completely forgot the initial goal.
|
|
52
|
+
|
|
53
|
+
### D. Environment & Tool Awareness
|
|
54
|
+
- 10 points: Accurately detected OS, terminal shell (e.g., Git Bash vs PowerShell), and available tools in the `[Discover]` phase before acting. No syntax errors caused by wrong shell assumptions.
|
|
55
|
+
- 5 points: Made an initial incorrect assumption about the terminal/tools but corrected it immediately after the first failure.
|
|
56
|
+
- 0 points: Blindly threw commands (e.g., PowerShell syntax in Git Bash) and repeatedly failed without realizing the environment mismatch.
|
|
57
|
+
|
|
58
|
+
## 3. Output Report `[Summarize]`
|
|
59
|
+
- **MANDATORY**: You MUST execute the evaluation script to generate and save the report.
|
|
60
|
+
- Find the `evaluate.js` script in the `scripts/` directory of this skill and run it via terminal:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
node "<this-skill-dir>/scripts/evaluate.js" <scoreA> <scoreB> <scoreC> <scoreD> "<Your insights here>"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
*(Example: `node scripts/evaluate.js 10 5 10 10 "The agent successfully used zoom-out..."`)* — run with `--help` for the scoring-dimension reference.
|
|
67
|
+
|
|
68
|
+
The script will automatically generate the Markdown-formatted **Scorecard Table** and save it to the project's `evals/` directory.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
const fs = require('fs');
|
|
3
|
+
const path = require('path');
|
|
4
|
+
let getWorkspaceRoot;
|
|
5
|
+
try {
|
|
6
|
+
({ getWorkspaceRoot } = require('../../scripts/lib/workspace'));
|
|
7
|
+
} catch (err) {
|
|
8
|
+
getWorkspaceRoot = () => null;
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
const args = process.argv.slice(2);
|
|
12
|
+
|
|
13
|
+
if (args[0] === '--help' || args[0] === '-h') {
|
|
14
|
+
console.log(`Score a Harness-guarded session and write an evaluation report.
|
|
15
|
+
|
|
16
|
+
Usage: node evaluate.js <scoreA> <scoreB> <scoreC> <scoreD> "<insights>"
|
|
17
|
+
|
|
18
|
+
Four positional scores (0, 5, or 10 each), in this order:
|
|
19
|
+
scoreA Correctness & Factuality
|
|
20
|
+
scoreB Token & Step Efficiency
|
|
21
|
+
scoreC Anti-loop & Focus
|
|
22
|
+
scoreD Environment & Tool Awareness
|
|
23
|
+
|
|
24
|
+
Followed by one free-text insights string (wrap in quotes).`);
|
|
25
|
+
process.exit(0);
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
if (args.length < 5) {
|
|
29
|
+
console.error("Usage: node evaluate.js <scoreA> <scoreB> <scoreC> <scoreD> \"<insights>\"");
|
|
30
|
+
console.error("Scores should be 0, 5, or 10.");
|
|
31
|
+
process.exit(1);
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
const [scoreA, scoreB, scoreC, scoreD, ...insightParts] = args;
|
|
35
|
+
const insights = insightParts.join(" ");
|
|
36
|
+
|
|
37
|
+
const scores = {
|
|
38
|
+
"Correctness & Factuality": parseInt(scoreA, 10),
|
|
39
|
+
"Token & Step Efficiency": parseInt(scoreB, 10),
|
|
40
|
+
"Anti-loop & Focus": parseInt(scoreC, 10),
|
|
41
|
+
"Environment & Tool Awareness": parseInt(scoreD, 10)
|
|
42
|
+
};
|
|
43
|
+
|
|
44
|
+
let total = 0;
|
|
45
|
+
let markdown = `## Harness Evaluation Report - ${new Date().toISOString()}\n\n| Dimension | Score |\n|---|---|\n`;
|
|
46
|
+
|
|
47
|
+
for (const [dim, score] of Object.entries(scores)) {
|
|
48
|
+
if (isNaN(score)) {
|
|
49
|
+
console.error(`Error: Score for ${dim} is not a valid number.`);
|
|
50
|
+
process.exit(1);
|
|
51
|
+
}
|
|
52
|
+
markdown += `| ${dim} | ${score}/10 |\n`;
|
|
53
|
+
total += score;
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
markdown += `| **Total Score** | **${total}/40** |\n\n`;
|
|
57
|
+
markdown += `### Insights\n${insights}\n\n---\n`;
|
|
58
|
+
|
|
59
|
+
// Store the evaluations in an evals folder in the project root
|
|
60
|
+
const workspaceRoot = getWorkspaceRoot();
|
|
61
|
+
if (!workspaceRoot) {
|
|
62
|
+
console.error('[eval-harness] Cannot write an evaluation report without a resolved git workspace.');
|
|
63
|
+
process.exit(1);
|
|
64
|
+
}
|
|
65
|
+
const evalDir = path.join(workspaceRoot, 'evals');
|
|
66
|
+
if (!fs.existsSync(evalDir)) {
|
|
67
|
+
fs.mkdirSync(evalDir, { recursive: true });
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
const reportPath = path.join(evalDir, 'evaluation-reports.md');
|
|
71
|
+
try {
|
|
72
|
+
fs.appendFileSync(reportPath, markdown, 'utf8');
|
|
73
|
+
console.log(`[Success] Evaluation report generated and saved to ${reportPath}`);
|
|
74
|
+
console.log('\n--- Report Preview ---\n');
|
|
75
|
+
console.log(markdown);
|
|
76
|
+
} catch (err) {
|
|
77
|
+
console.error(`[Error] Failed to write evaluation report: ${err.message}`);
|
|
78
|
+
process.exit(1);
|
|
79
|
+
}
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: fable-discipline
|
|
3
|
+
description: Enforce fable-mode's context discipline as a shadow guard. Use when running staged multi-agent loops to prevent context bloat, cap per-stage output size, keep each stage within its physical token boundaries, and stop agents from reading files outside the current stage's scope.
|
|
4
|
+
license: Apache-2.0
|
|
5
|
+
metadata:
|
|
6
|
+
author: Miya Daniel
|
|
7
|
+
version: 0.4.0
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Fable Discipline (Macro Task Discipline & Safety Net)
|
|
11
|
+
|
|
12
|
+
## 📋 Skill Contract
|
|
13
|
+
|
|
14
|
+
| Component | Specification |
|
|
15
|
+
| :--- | :--- |
|
|
16
|
+
| **Trigger / Input** | Background shadow for the whole duration `fable-mode` is active — not a standalone one-shot trigger. |
|
|
17
|
+
| **Expected Output** | State compaction after milestones; one Git commit per independent logic block; a state manifest at every sub-agent handoff. |
|
|
18
|
+
| **State Mutations** | None of its own — constrains how `fable-mode` and `multi-agent-workspace` mutate state. |
|
|
19
|
+
| **Enforcement Gate** | **HALTS execution immediately** if build errors diverge (more fixes → more breakage); forces `zoom-out` instead of continued patching. |
|
|
20
|
+
|
|
21
|
+
Shadow guard for `fable-mode`: MUST run in the background for its whole duration.
|
|
22
|
+
|
|
23
|
+
## USE FOR:
|
|
24
|
+
- Long architectural tasks under `fable-mode`
|
|
25
|
+
- Compacting state after milestones
|
|
26
|
+
- Sub-agent handoffs via `multi-agent-workspace`
|
|
27
|
+
- Halting on diverging build errors
|
|
28
|
+
|
|
29
|
+
## DO NOT USE FOR:
|
|
30
|
+
- Small tasks where `fable-mode` is not active
|
|
31
|
+
- Standalone one-shot invocation (background shadow skill)
|
|
32
|
+
- Routine single-file edits
|
|
33
|
+
|
|
34
|
+
## Core Rules
|
|
35
|
+
|
|
36
|
+
1. **Anti-context bloat**: After each milestone, compact state and decisions; drop unneeded history. No broad regex without precise conditions; no reading irrelevant files over 1000 lines.
|
|
37
|
+
2. **Physical boundaries**: Know your CWD before touching core architecture; one Git commit per independent logic block.
|
|
38
|
+
3. **Agent handoff**: Outgoing agents MUST leave a state manifest (completed APIs, expected inputs/outputs); incoming agents MUST verify it first.
|
|
39
|
+
4. **Circuit breaker**: If build errors diverge, HALT, call `zoom-out`, and map the error dependency graph for the human.
|
|
40
|
+
|
|
41
|
+
Deep dive: <this-skill-dir>/references/discipline-rules.md
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# Fable Discipline — Full Detail
|
|
2
|
+
|
|
3
|
+
This skill acts as the shadow guard for `fable-mode`. As long as `fable-mode` is active, this skill MUST run in the background to prevent large tasks from spiraling out of control.
|
|
4
|
+
|
|
5
|
+
## 1. Anti-Context Bloat Defense
|
|
6
|
+
When executing large architectural tasks, conversation logs expand rapidly, leading to model attention loss (hallucinations or forgetting the original intent).
|
|
7
|
+
- **Strategic Compact**: After completing each core milestone, forcefully summarize the current state and decisions, discarding unnecessary past conversation details.
|
|
8
|
+
- **Avoid Broad Reads**: Prohibited from using broad Regex searches without precise conditions or reading irrelevant files over 1000 lines.
|
|
9
|
+
|
|
10
|
+
## 2. Strict Physical Boundaries
|
|
11
|
+
- **Environment Isolation**: Before modifying any core architecture, ensure you are clearly aware of the Current Working Directory (CWD).
|
|
12
|
+
- **Atomic Commits**: You are NOT allowed to modify dozens of files at once before testing. For every independently functioning logic block completed, advise the human to create a Git Commit as a safe save point.
|
|
13
|
+
|
|
14
|
+
## 3. Agent Handoff Protocol
|
|
15
|
+
When `fable-mode` spawns and switches between different sub-agents via `multi-agent-workspace`, strict handoff discipline must be observed:
|
|
16
|
+
- **State Manifest**: The previous agent MUST leave a clear state record (e.g., what APIs were completed, expected inputs/outputs).
|
|
17
|
+
- **Contract Testing**: The very first step for the succeeding agent is to verify if the state manifest left by the previous agent is correct.
|
|
18
|
+
|
|
19
|
+
## 4. Ultimate Circuit Breaker: Indefinite Halt
|
|
20
|
+
- Large tasks easily generate the sunk cost fallacy of "I think it's almost fixed, let me try one more time."
|
|
21
|
+
- If the system's Build Errors exhibit a divergent trend (the more you fix, the more it breaks), **HALT EXECUTION IMMEDIATELY**.
|
|
22
|
+
- Call `zoom-out`, organize the error dependency graph for the human, and seek architectural-level refactoring advice.
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
# Fable Stage Contract Format
|
|
2
|
+
|
|
3
|
+
Issue #85 Phase 3 makes the stage map a machine-readable execution contract.
|
|
4
|
+
The router still chooses only the execution topology. Fable owns the stage map,
|
|
5
|
+
model selector, worker briefs, stage audit, replan budget, and cold verifier.
|
|
6
|
+
Phase 4 adds an optional bounded ensemble modifier without creating a new base
|
|
7
|
+
topology.
|
|
8
|
+
|
|
9
|
+
Run state is stored under the existing workspace-keyed global Harness state
|
|
10
|
+
root, never inside the repository:
|
|
11
|
+
|
|
12
|
+
```text
|
|
13
|
+
<workspace-state>/state/fable-runs/<runId>/
|
|
14
|
+
run.json
|
|
15
|
+
contracts/<stageId>.json
|
|
16
|
+
evidence/<stageId>.json
|
|
17
|
+
ensemble.json # only when ensemble review executes
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
`planId` is a deterministic content hash of the router contract. `runId` is
|
|
21
|
+
unique per orchestration. Two runs of the same plan therefore share a `planId`
|
|
22
|
+
but never share mutable stage-contract files.
|
|
23
|
+
|
|
24
|
+
## Stage schema v2
|
|
25
|
+
|
|
26
|
+
Every planned stage declares dependency and write scope before dispatch:
|
|
27
|
+
|
|
28
|
+
```json
|
|
29
|
+
{
|
|
30
|
+
"stageId": "stage-3",
|
|
31
|
+
"goal": "implement one bounded objective",
|
|
32
|
+
"agent": "fable-worker-sonnet",
|
|
33
|
+
"task": "self-contained task statement",
|
|
34
|
+
"inputs": ["relevant files", "upstream outputs"],
|
|
35
|
+
"expectedOutputs": ["artifact or evidence"],
|
|
36
|
+
"outputPath": "path/to/artifact-or-null",
|
|
37
|
+
"dependsOn": ["stage-1"],
|
|
38
|
+
"writeSet": ["src/auth", "tests/auth.test.js"],
|
|
39
|
+
"checkCommand": "node --test tests/auth.test.js",
|
|
40
|
+
"passCondition": "exit 0",
|
|
41
|
+
"failureReturn": "failure summary + evidence + missing prerequisite"
|
|
42
|
+
}
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Rules:
|
|
46
|
+
|
|
47
|
+
- `stageId`, `goal`, `agent`, and `task` are required.
|
|
48
|
+
- `dependsOn` is always explicit. Use `[]` for a root stage.
|
|
49
|
+
- `writeSet` is always explicit. Use `[]` for a read-only stage.
|
|
50
|
+
- `writeSet` contains concrete repository-relative path scopes, not globs,
|
|
51
|
+
absolute paths, or `..` traversal.
|
|
52
|
+
- a directory scope covers descendants; overlapping scopes cannot run in the
|
|
53
|
+
same parallel batch.
|
|
54
|
+
- every `dependsOn` target must exist; self edges and cycles are invalid.
|
|
55
|
+
- `checkCommand` is exact-match evidence when present. `passCondition` explains
|
|
56
|
+
what that command proves; it does not replace the command result.
|
|
57
|
+
- workers do not widen their own `writeSet`. If scope must grow, return to the
|
|
58
|
+
orchestrator and produce a revised plan before dispatch.
|
|
59
|
+
|
|
60
|
+
## Workflow-plan consumer
|
|
61
|
+
|
|
62
|
+
Before dispatch, write the router contract and stage array to files and run:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
node fable-mode/scripts/workflow-plan-consumer.js \
|
|
66
|
+
--plan-file <router-contract.json> \
|
|
67
|
+
--stages-file <stages.json> \
|
|
68
|
+
--root <workspace> \
|
|
69
|
+
--run-id <optional-stable-run-label> \
|
|
70
|
+
--session-id <optional-host-session-id>
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The consumer validates the graph, derives `planId`, creates the isolated run
|
|
74
|
+
root, and emits execution batches. It consumes only router-owned decisions:
|
|
75
|
+
strategy, parallelism, verifier requirement, workspace/memory hints, limits,
|
|
76
|
+
and requested-model metadata. It does **not** choose a model or execute a
|
|
77
|
+
stage.
|
|
78
|
+
|
|
79
|
+
For `fable-parallel`, every ready batch must have resolved dependencies and
|
|
80
|
+
pairwise non-overlapping write sets. An invalid graph or overlap rejects the
|
|
81
|
+
plan before workers are spawned. For `fable-staged` and
|
|
82
|
+
`fable-multi-agent-workspace`, the same dependency graph is serialized unless
|
|
83
|
+
a later validated plan explicitly allows parallelism.
|
|
84
|
+
|
|
85
|
+
## Bounded ensemble evidence
|
|
86
|
+
|
|
87
|
+
`workflowPlan.ensemble` is a modifier on a selected Fable topology, not a sixth
|
|
88
|
+
execution strategy. The kernel may add it only for high-uncertainty/high-stakes
|
|
89
|
+
work with comparable outputs (choice/ranking/verdict/recommendation) or when
|
|
90
|
+
preserving disagreement is itself the deliverable. Creative generation,
|
|
91
|
+
mechanical bulk work, explicit `no ensemble`, unavailable subagents, and blocked
|
|
92
|
+
base plans do not receive the modifier.
|
|
93
|
+
|
|
94
|
+
Current contract:
|
|
95
|
+
|
|
96
|
+
```json
|
|
97
|
+
{
|
|
98
|
+
"maxCandidates": 3,
|
|
99
|
+
"diversity": ["model", "prompt", "evidence"],
|
|
100
|
+
"synthesis": "preserve-disagreement",
|
|
101
|
+
"verifier": "independent",
|
|
102
|
+
"reasonCodes": ["ensemble-high-uncertainty-comparable-output"]
|
|
103
|
+
}
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Each candidate passed to `fable-mode/scripts/ensemble-review.js` records a
|
|
107
|
+
bounded position, a model/prompt/evidence diversity fingerprint, objective
|
|
108
|
+
verification status/evidence, and optional correctness/cost observations.
|
|
109
|
+
Candidates with the same model+prompt+evidence fingerprint are not independent
|
|
110
|
+
and are rejected. The verifier uses a distinct identity, declares
|
|
111
|
+
`independent=true`, and supplies its own evidence.
|
|
112
|
+
|
|
113
|
+
The synthesizer keeps every position and its evidence. A strict majority is
|
|
114
|
+
only a proposal: any objective verification failure on the proposed position
|
|
115
|
+
blocks selection, and verifier rejection/inconclusive/mismatch also blocks it.
|
|
116
|
+
An accepted position therefore never deletes minority positions or evidence
|
|
117
|
+
gaps.
|
|
118
|
+
|
|
119
|
+
Run it after the candidate artifacts/checks exist:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
node fable-mode/scripts/ensemble-review.js \
|
|
123
|
+
--plan-file <router-contract.json> \
|
|
124
|
+
--candidates-file <candidates.json> \
|
|
125
|
+
--verifier-file <verifier.json> \
|
|
126
|
+
--root <workspace> \
|
|
127
|
+
--run-id <runId> \
|
|
128
|
+
[--paired-evidence-file <paired-benchmark.json>]
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
The correlated result is written to `ensemble.json` under the same run root.
|
|
132
|
+
Correctness is recorded separately from tool calls, tokens, elapsed time,
|
|
133
|
+
retries, and failure mode so #71/#83 can consume the evidence without
|
|
134
|
+
conflating quality with cost. Missing telemetry is allowed and never changes
|
|
135
|
+
the strategy/result. No ensemble or multi-agent improvement claim is permitted
|
|
136
|
+
unless paired evidence establishes the same engine/model/fixture/rubric/config,
|
|
137
|
+
a sample count, effect estimate, and uncertainty.
|
|
138
|
+
|
|
139
|
+
## Verification evidence
|
|
140
|
+
|
|
141
|
+
`contract-test.js` watches Bash/PowerShell calls and resolves an exact
|
|
142
|
+
`checkCommand` only when it can correlate the command to one run/stage. The
|
|
143
|
+
result updates the stage contract and writes:
|
|
144
|
+
|
|
145
|
+
```text
|
|
146
|
+
evidence/<stageId>.json
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
with `planId`, `runId`, `stageId`, command, exit code, status, and captured
|
|
150
|
+
evidence. If the same command is pending in multiple uncorrelated runs, the
|
|
151
|
+
hook updates none of them and reports the ambiguity.
|
|
152
|
+
|
|
153
|
+
Before delivery, every check-bearing contract in the run must be `pass` (not
|
|
154
|
+
`pending`, `planned`, `running`, or `fail`). A failed check returns to the
|
|
155
|
+
orchestrator for bounded re-planning.
|
|
156
|
+
|
|
157
|
+
## Scope audit
|
|
158
|
+
|
|
159
|
+
`subagent-scope-guard.js` snapshots active stage write sets at the beginning of
|
|
160
|
+
a worker burst and compares the post-burst Git diff paths with that immutable
|
|
161
|
+
snapshot. Expected paths are attributed to the matching stage; worker-level
|
|
162
|
+
attribution is added when the host exposes a worker/subagent id and the stage
|
|
163
|
+
contract records it. Ambiguous or out-of-scope changes exit 2. With no declared
|
|
164
|
+
stage contracts, the older conservative behavior remains: every newly changed
|
|
165
|
+
path requires review.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: fable-mode
|
|
3
|
+
description: "Stage large, multi-source or multi-session tasks through a written plan, named fable agents, failable per-stage checks, and skeptical delivery review; use fable-opus, fable-sonnet, or fable-haiku for explicit model selection."
|
|
4
|
+
license: Apache-2.0
|
|
5
|
+
metadata:
|
|
6
|
+
author: Miya Daniel
|
|
7
|
+
version: 0.4.0
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Fable Mode (v3)
|
|
11
|
+
|
|
12
|
+
## Skill Contract
|
|
13
|
+
|
|
14
|
+
| Component | Specification |
|
|
15
|
+
| :--- | :--- |
|
|
16
|
+
| **Trigger / Input** | Tier 3 work spanning multiple files, sources, or sessions. |
|
|
17
|
+
| **Expected Output** | Stage map, named-agent briefs, failable checks, and skeptical review. |
|
|
18
|
+
| **State Mutations** | Native host TODO tracker or a Markdown checklist; one JSON audit record per stage. |
|
|
19
|
+
| **Enforcement Gate** | `<this-skill-dir>/scripts/model-selector.js`, stage contracts, and `<skills-repo-root>/harness-everything/scripts/verify-gate.js`. |
|
|
20
|
+
|
|
21
|
+
## USE FOR:
|
|
22
|
+
- Large multi-file, multi-source, or multi-session work
|
|
23
|
+
- Explicit `fable on haiku`, `fable on sonnet`, or `fable on opus` requests
|
|
24
|
+
- Work needing named delegation and cold verification
|
|
25
|
+
|
|
26
|
+
## DO NOT USE FOR:
|
|
27
|
+
- One obvious single-pass edit
|
|
28
|
+
- Ordinary Tier 2 implementation or bugfix work
|
|
29
|
+
|
|
30
|
+
## Workflow
|
|
31
|
+
|
|
32
|
+
1. Discover the runtime and lock the authorized file scope before edits.
|
|
33
|
+
2. Write a numbered stage map with one artifact and pass condition per stage; allow at most two full replans.
|
|
34
|
+
3. Resolve the requested model with `model-selector.js`; never silently downgrade.
|
|
35
|
+
4. Delegate by named agent where available: Opus orchestrates, Sonnet reasons, Haiku handles mechanical work, and workers never spawn workers.
|
|
36
|
+
5. Write a stage contract, run its named check, then cold-review high-stakes artifacts with `fable-verifier`.
|
|
37
|
+
6. Record requested/effective model, fallback reason, stage brief, pass condition, verification command, and verifier result for every stage. Escalate unresolved blockers.
|
|
38
|
+
|
|
39
|
+
Deep dive: <this-skill-dir>/references/model-matrix.md
|