harness-everything 0.2.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +20 -0
- package/.claude-plugin/marketplace.json +27 -0
- package/.claude-plugin/plugin.json +52 -0
- package/.gitattributes +1 -0
- package/.github/workflows/behavioral-evals.yml +36 -0
- package/.github/workflows/ci.yml +122 -0
- package/.github/workflows/release.yml +71 -0
- package/.husky/commit-msg +1 -0
- package/.releaserc.json +113 -0
- package/.waza.yaml +22 -0
- package/AGENTS.md +71 -2
- package/BENCHMARK_SOP.md +82 -62
- package/CHANGELOG.md +488 -114
- package/CODE_OF_CONDUCT.md +24 -0
- package/CONTRIBUTING.md +60 -0
- package/Dockerfile +18 -18
- package/LICENSE +201 -201
- package/PRIVACY.md +25 -0
- package/README.md +310 -197
- package/RELEASING.md +82 -0
- package/SECURITY.md +33 -0
- package/SUPPORT.md +27 -0
- package/TERMS.md +21 -0
- package/VERIFICATION.md +283 -347
- package/behavioral-evals/README.md +106 -0
- package/behavioral-evals/analyze-plugin-results.js +110 -0
- package/behavioral-evals/case-validator.js +194 -0
- package/behavioral-evals/cases/baseline-add-feature.yaml +34 -0
- package/behavioral-evals/cases/baseline-api-design.yaml +32 -0
- package/behavioral-evals/cases/baseline-debugging.yaml +31 -0
- package/behavioral-evals/cases/baseline-dependency.yaml +24 -0
- package/behavioral-evals/cases/baseline-documentation.yaml +34 -0
- package/behavioral-evals/cases/baseline-environment-detection.yaml +21 -0
- package/behavioral-evals/cases/baseline-error-handling.yaml +31 -0
- package/behavioral-evals/cases/baseline-eval-harness.yaml +27 -0
- package/behavioral-evals/cases/baseline-fable-discipline.yaml +31 -0
- package/behavioral-evals/cases/baseline-find-skills-exposure.yaml +49 -0
- package/behavioral-evals/cases/baseline-find-skills.yaml +21 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +33 -0
- package/behavioral-evals/cases/baseline-install-cognitive-os.yaml +24 -0
- package/behavioral-evals/cases/baseline-multi-agent-workspace.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance-monitoring.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance.yaml +34 -0
- package/behavioral-evals/cases/baseline-refactor-with-tests.yaml +35 -0
- package/behavioral-evals/cases/baseline-refactor.yaml +27 -0
- package/behavioral-evals/cases/baseline-rewrite-commits.yaml +34 -0
- package/behavioral-evals/cases/baseline-security-review.yaml +28 -0
- package/behavioral-evals/cases/baseline-self-evolve.yaml +31 -0
- package/behavioral-evals/cases/baseline-simple-bugfix.yaml +33 -0
- package/behavioral-evals/cases/baseline-skill-creator.yaml +20 -0
- package/behavioral-evals/cases/baseline-skill-style.yaml +21 -0
- package/behavioral-evals/cases/baseline-tdd-workflow.yaml +28 -0
- package/behavioral-evals/cases/baseline-to-spec.yaml +34 -0
- package/behavioral-evals/cases/baseline-to-tickets.yaml +24 -0
- package/behavioral-evals/cases/baseline-todo-driven-workflow.yaml +32 -0
- package/behavioral-evals/cases/baseline-using-git-worktrees.yaml +15 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +50 -0
- package/behavioral-evals/cases/pressure-budget-skip-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-bypass-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +37 -0
- package/behavioral-evals/cases/pressure-expert-skip-testing.yaml +31 -0
- package/behavioral-evals/cases/pressure-fatigue-skip.yaml +33 -0
- package/behavioral-evals/cases/pressure-management-skip-all.yaml +28 -0
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +33 -0
- package/behavioral-evals/cases/pressure-skip-docs.yaml +50 -0
- package/behavioral-evals/cases/pressure-skip-error-handling.yaml +35 -0
- package/behavioral-evals/cases/pressure-skip-security.yaml +32 -0
- package/behavioral-evals/cases/pressure-skip-tests.yaml +27 -0
- package/behavioral-evals/cases/pressure-skip-verification.yaml +43 -0
- package/behavioral-evals/cases/pressure-social-skip-review.yaml +28 -0
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +50 -0
- package/behavioral-evals/cases/scope-discipline.yaml +33 -0
- package/behavioral-evals/cases/tier1-no-overplan.yaml +26 -0
- package/behavioral-evals/cases/verify-before-done.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/historical-case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/live-run-blocker.md +16 -0
- package/behavioral-evals/evidence/2026-09-07/manifest.json +88 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/historical-case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/historical-case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/case.yaml +30 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/historical-case.yaml +23 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.json +426 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.md +24 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/case.yaml +19 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/result.sanitized.json +480 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/transcript.sanitized.jsonl +95 -0
- package/behavioral-evals/evidence-tool.js +443 -0
- package/behavioral-evals/fixtures/claude-denied.jsonl +3 -0
- package/behavioral-evals/fixtures/claude-legacy.json +1 -0
- package/behavioral-evals/fixtures/claude-oral-only.jsonl +2 -0
- package/behavioral-evals/fixtures/claude-success.jsonl +5 -0
- package/behavioral-evals/fixtures/claude-truncated.jsonl +3 -0
- package/behavioral-evals/fixtures/opencode-success.jsonl +6 -0
- package/behavioral-evals/results/.gitkeep +0 -0
- package/behavioral-evals/results/2026-08-23-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-skip-verification.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-sunk-cost-retry.json +20 -0
- package/behavioral-evals/results/2026-08-23-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-23-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-23-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-api-design.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-debugging.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-dependency.json +16 -0
- package/behavioral-evals/results/2026-08-27-baseline-documentation.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-git-commit.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance-monitoring.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor-with-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-security-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-simple-bugfix.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-tdd-workflow.json +24 -0
- package/behavioral-evals/results/2026-08-27-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-budget-skip-review.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-bypass-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-complexity-skip.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-expert-skip-testing.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-fatigue-skip.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-management-skip-all.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-docs.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-security.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-verification.json +24 -0
- package/behavioral-evals/results/2026-08-27-pressure-social-skip-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-sunk-cost-retry.json +24 -0
- package/behavioral-evals/results/2026-08-27-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-27-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-27-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-31-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-09-03-baseline-environment-detection-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-eval-harness-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-fable-discipline-pair.json +149 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-exposure-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-install-cognitive-os-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-multi-agent-workspace-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-rewrite-commits-pair.json +95 -0
- package/behavioral-evals/results/2026-09-03-baseline-self-evolve-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-creator-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-style-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-spec-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-tickets-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-todo-driven-workflow-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-using-git-worktrees-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-summary-both.json +41 -0
- package/behavioral-evals/results/2026-09-04-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-04-baseline-self-evolve-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-baseline-skill-creator-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-summary-both.json +41 -0
- package/behavioral-evals/results/run-20260827-115509.log +287 -0
- package/behavioral-evals/run-with-plugin.js +300 -0
- package/behavioral-evals/run.js +928 -0
- package/behavioral-evals/transcript-parser.js +591 -0
- package/benchmarks/README.md +63 -0
- package/benchmarks/ab-test-2026-08-26/results.json +55 -0
- package/benchmarks/results/logs/.gitkeep +0 -0
- package/benchmarks/results/logs/test-a-harness.md +31 -0
- package/benchmarks/results/logs/test-a-vanilla.md +28 -0
- package/benchmarks/results/logs/test-b-micro-loop-harness.md +48 -0
- package/benchmarks/results/logs/test-b-micro-loop-vanilla.md +36 -0
- package/benchmarks/results/logs/test-c-attention-loss-harness.md +40 -0
- package/benchmarks/results/logs/test-c-attention-loss-vanilla.md +63 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-harness.md +24 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-vanilla.md +29 -0
- package/benchmarks/results/logs/test-e-shell-awareness-harness.md +20 -0
- package/benchmarks/results/logs/test-e-shell-awareness-vanilla.md +20 -0
- package/benchmarks/results/logs/test-f-fact-audit-harness.md +16 -0
- package/benchmarks/results/logs/test-f-fact-audit-vanilla.md +16 -0
- package/benchmarks/results/results.json +158 -0
- package/benchmarks/run.js +220 -0
- package/benchmarks/schema.json +44 -0
- package/bin/cli.js +164 -97
- package/ci/ab-test-cases/grill-me-adversarial.yaml +28 -0
- package/ci/ab-test-cases/tdd-test-first.yaml +30 -0
- package/ci/ab-test-cases/verification-loop-gates.yaml +38 -0
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +19 -0
- package/ci/ab-test-harness.js +485 -0
- package/ci/action-gate-platform-doc.test.js +31 -0
- package/ci/action-gate.test.js +260 -0
- package/{eval-framework → ci}/cases/case1.json +5 -5
- package/ci/cases/case10-small-feature.json +5 -0
- package/ci/cases/case11-readme-typo-update.json +5 -0
- package/ci/cases/case12-chinese-macro-audit.json +5 -0
- package/ci/cases/case2.json +5 -0
- package/{eval-framework → ci}/cases/case3.json +5 -5
- package/{eval-framework → ci}/cases/case4-zh-tier2.json +5 -5
- package/{eval-framework → ci}/cases/case5-zh-tier3.json +5 -5
- package/ci/cases/case6-macro-skill-audit.json +5 -0
- package/ci/cases/case7-macro-reference-audit.json +5 -0
- package/ci/cases/case8-controlled-ab-benchmark.json +5 -0
- package/ci/cases/case9-four-issue-audit.json +5 -0
- package/ci/consistency-check.js +393 -0
- package/ci/description-collision.js +118 -0
- package/ci/disclosure-check.js +225 -0
- package/ci/doc-capability-consistency.test.js +180 -0
- package/ci/fable-mode-test.js +120 -0
- package/ci/fixtures/agency-agents/divisions.json +6 -0
- package/ci/fixtures/agency-agents/engineering/code-reviewer.md +11 -0
- package/ci/fixtures/agency-agents/testing/reality-checker.md +11 -0
- package/ci/fixtures/bad-frontmatter-colon/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-a/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-b/SKILL.md +16 -0
- package/ci/fixtures/phase4/parallel-disjoint.stages.json +32 -0
- package/ci/fixtures/phase4/parallel-read-only.stages.json +32 -0
- package/ci/installer-roundtrip.test.js +159 -0
- package/ci/invariant-routing.test.js +74 -0
- package/ci/issue99-runtime.test.js +218 -0
- package/ci/mechanism-2a-rule-of-3.test.js +46 -0
- package/ci/mechanism-2aa-codex-action-gate-policy.test.js +117 -0
- package/ci/mechanism-2ab-openai-plugin-sync-integrity.test.js +80 -0
- package/ci/mechanism-2b-boundary-guard.test.js +16 -0
- package/ci/mechanism-2c-state-persist.test.js +29 -0
- package/ci/mechanism-2d-fact-audit.test.js +18 -0
- package/ci/mechanism-2e-scope-guard.test.js +24 -0
- package/ci/mechanism-2f-stop-gate.test.js +23 -0
- package/ci/mechanism-2g-platform-ignore.test.js +88 -0
- package/ci/mechanism-2h-installer-manifest.test.js +147 -0
- package/ci/mechanism-2i-crlf-frontmatter.test.js +145 -0
- package/ci/mechanism-2i-install-verification.test.js +34 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +62 -0
- package/ci/mechanism-2k-reference-check.test.js +75 -0
- package/ci/mechanism-2l-multi-agent-workspace.test.js +301 -0
- package/ci/mechanism-2m-self-repo-guard.test.js +65 -0
- package/ci/mechanism-2n-opencode-plugin.test.js +461 -0
- package/ci/mechanism-2o-state-home-isolation.test.js +183 -0
- package/ci/mechanism-2p-installer-links.test.js +183 -0
- package/ci/mechanism-2q-installer-global-shared-store.test.js +132 -0
- package/ci/mechanism-2r-uninstall-flags.test.js +148 -0
- package/ci/mechanism-2s-tdd-quality.test.js +203 -0
- package/ci/mechanism-2t-disclosure.test.js +90 -0
- package/ci/mechanism-2t-evidence-triage.test.js +200 -0
- package/ci/mechanism-2t-issue-20.test.js +75 -0
- package/ci/mechanism-2t-platform-skill-targets.test.js +164 -0
- package/ci/mechanism-2t-transcript-parser.test.js +179 -0
- package/ci/mechanism-2t-workspace-identity.test.js +200 -0
- package/ci/mechanism-2u-issue-42-regressions.test.js +239 -0
- package/ci/mechanism-2u-issue-44.test.js +35 -0
- package/ci/mechanism-2v-verifier-boundaries.test.js +138 -0
- package/ci/mechanism-2w-platform-compatibility.test.js +148 -0
- package/ci/mechanism-2x-release-automation.test.js +157 -0
- package/ci/mechanism-2x-release-commit-message.test.js +66 -0
- package/ci/mechanism-2y-release-tooling-compat.test.js +47 -0
- package/ci/mechanism-2z-claude-plugin-manifest.test.js +29 -0
- package/ci/mechanism-test.js +71 -0
- package/ci/mutation-check.js +53 -0
- package/ci/mutation-loader.js +57 -0
- package/ci/negative-controls.js +114 -0
- package/ci/openai-plugin-package.test.js +182 -0
- package/ci/openai-plugin-runtime.test.js +175 -0
- package/ci/openai-plugin-submission.test.js +101 -0
- package/ci/phase4-ensemble-memory.test.js +307 -0
- package/ci/reference-check.js +160 -0
- package/ci/release-consistency-check.js +146 -0
- package/ci/router-phase2-safety.test.js +81 -0
- package/ci/router-workflow-plan.test.js +264 -0
- package/ci/routing-check.js +91 -0
- package/{eval-framework → ci}/runner.js +57 -56
- package/ci/skill-routing-check.js +135 -0
- package/ci/test-behavioral-case-validation.js +62 -0
- package/ci/test-helper.js +114 -0
- package/ci/test-negative-controls.js +148 -0
- package/ci/workflow-plan-consumers-edge.test.js +72 -0
- package/ci/workflow-plan-consumers.test.js +276 -0
- package/commitlint.config.js +25 -0
- package/docker-verify.sh +266 -266
- package/docs/architecture.md +179 -95
- package/docs/audit.md +55 -0
- package/docs/codex-action-gate.md +90 -0
- package/docs/issue-20-rollup.md +47 -0
- package/docs/mechanism-first-skill-mesh.md +165 -0
- package/docs/openai-plugin.md +118 -0
- package/docs/philosophy.md +58 -45
- package/docs/platform-capabilities.md +73 -0
- package/docs/platform-compatibility.json +270 -0
- package/docs/reflection.md +63 -34
- package/docs/release-evidence.md +25 -0
- package/docs/routing.md +171 -45
- package/docs/security-audit.md +38 -0
- package/docs/troubleshooting.md +320 -0
- package/docs/verifier-boundaries.md +45 -0
- package/docs/workflows/environment-detection.md +10 -10
- package/docs/workflows/eval-harness.md +13 -6
- package/docs/workflows/fable-discipline.md +9 -9
- package/docs/workflows/fable-mode.md +85 -32
- package/docs/workflows/find-skills.md +76 -0
- package/docs/workflows/git-commit.md +18 -9
- package/docs/workflows/grill-me.md +11 -11
- package/docs/workflows/grill-with-docs.md +14 -7
- package/docs/workflows/harness-everything.md +95 -92
- package/docs/workflows/improve-codebase-architecture.md +15 -7
- package/docs/workflows/install-cognitive-os.md +60 -60
- package/docs/workflows/multi-agent-workspace.md +54 -0
- package/docs/workflows/repo-docs.md +12 -7
- package/docs/workflows/security-review.md +26 -7
- package/docs/workflows/self-evolve.md +53 -37
- package/docs/workflows/skill-creator.md +80 -75
- package/docs/workflows/skill-style.md +64 -64
- package/docs/workflows/tdd.md +78 -39
- package/docs/workflows/to-spec.md +83 -0
- package/docs/workflows/to-tickets.md +80 -0
- package/docs/workflows/todo-driven-workflow.md +67 -36
- package/docs/workflows/using-git-worktrees.md +12 -8
- package/docs/workflows/verification-loop.md +16 -14
- package/docs/workflows/verify-before-claim.md +12 -8
- package/docs/workflows/zoom-out.md +22 -10
- package/environment-detection/SKILL.md +25 -61
- package/environment-detection/references/shell-syntax-rules.md +76 -0
- package/environment-detection/scripts/preflight.js +103 -93
- package/eval-framework/fixtures/bad-frontmatter-colon/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-a/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-b/SKILL.md +14 -0
- package/eval-framework/fixtures/missing-delimiter/SKILL.md +13 -0
- package/eval-framework/fixtures/over-budget-skill/SKILL.md +36 -0
- package/eval-harness/SKILL.md +34 -51
- package/eval-harness/references/scoring-rubric.md +68 -0
- package/eval-harness/scripts/evaluate.js +79 -61
- package/evals/environment-detection/eval.yaml +26 -0
- package/evals/environment-detection/tasks/negative-trigger.yaml +16 -0
- package/evals/environment-detection/tasks/positive-trigger.yaml +15 -0
- package/evals/eval-harness/eval.yaml +26 -0
- package/evals/eval-harness/tasks/negative-trigger.yaml +16 -0
- package/evals/eval-harness/tasks/positive-trigger.yaml +15 -0
- package/evals/evaluation-reports.md +14 -0
- package/evals/fable-discipline/eval.yaml +26 -0
- package/evals/fable-discipline/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-discipline/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-mode/eval.yaml +26 -0
- package/evals/fable-mode/tasks/negative-tier2-no-model.yaml +18 -0
- package/evals/fable-mode/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-mode/tasks/positive-haiku-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-opus-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-sonnet-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-trigger.yaml +15 -0
- package/evals/find-skills/eval.yaml +26 -0
- package/evals/find-skills/tasks/negative-trigger.yaml +16 -0
- package/evals/find-skills/tasks/positive-trigger.yaml +15 -0
- package/evals/git-commit/eval.yaml +26 -0
- package/evals/git-commit/tasks/negative-trigger.yaml +16 -0
- package/evals/git-commit/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-me/eval.yaml +26 -0
- package/evals/grill-me/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-me/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-with-docs/eval.yaml +26 -0
- package/evals/grill-with-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-with-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/harness-everything/eval.yaml +26 -0
- package/evals/harness-everything/tasks/negative-non-software.yaml +16 -0
- package/evals/harness-everything/tasks/negative-skill-indicated.yaml +15 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier2.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier3.yaml +16 -0
- package/evals/improve-codebase-architecture/eval.yaml +26 -0
- package/evals/improve-codebase-architecture/tasks/negative-trigger.yaml +16 -0
- package/evals/improve-codebase-architecture/tasks/positive-trigger.yaml +15 -0
- package/evals/install-cognitive-os/eval.yaml +26 -0
- package/evals/install-cognitive-os/tasks/negative-trigger.yaml +16 -0
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/eval.yaml +26 -0
- package/evals/multi-agent-workspace/tasks/negative-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +15 -0
- package/evals/repo-docs/eval.yaml +26 -0
- package/evals/repo-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/repo-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/rewrite-commits/eval.yaml +26 -0
- package/evals/rewrite-commits/tasks/negative-trigger.yaml +16 -0
- package/evals/rewrite-commits/tasks/positive-trigger.yaml +15 -0
- package/evals/security-review/eval.yaml +26 -0
- package/evals/security-review/tasks/negative-trigger.yaml +16 -0
- package/evals/security-review/tasks/positive-trigger.yaml +15 -0
- package/evals/self-evolve/eval.yaml +26 -0
- package/evals/self-evolve/tasks/negative-trigger.yaml +16 -0
- package/evals/self-evolve/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-creator/eval.yaml +26 -0
- package/evals/skill-creator/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-creator/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-style/eval.yaml +26 -0
- package/evals/skill-style/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-style/tasks/positive-trigger.yaml +15 -0
- package/evals/tdd/eval.yaml +26 -0
- package/evals/tdd/tasks/negative-trivia.yaml +16 -0
- package/evals/tdd/tasks/positive-bugfix.yaml +15 -0
- package/evals/tdd/tasks/positive-feature.yaml +16 -0
- package/evals/tdd/tasks/positive-refactor.yaml +15 -0
- package/evals/to-spec/eval.yaml +26 -0
- package/evals/to-spec/tasks/negative-trigger.yaml +16 -0
- package/evals/to-spec/tasks/positive-trigger.yaml +15 -0
- package/evals/to-tickets/eval.yaml +26 -0
- package/evals/to-tickets/tasks/negative-trigger.yaml +15 -0
- package/evals/to-tickets/tasks/positive-trigger.yaml +15 -0
- package/evals/todo-driven-workflow/eval.yaml +26 -0
- package/evals/todo-driven-workflow/tasks/negative-single-edit.yaml +16 -0
- package/evals/todo-driven-workflow/tasks/positive-multi-step.yaml +15 -0
- package/evals/todo-driven-workflow/tasks/positive-refactor-flow.yaml +15 -0
- package/evals/using-git-worktrees/eval.yaml +26 -0
- package/evals/using-git-worktrees/tasks/negative-trigger.yaml +15 -0
- package/evals/using-git-worktrees/tasks/positive-trigger.yaml +15 -0
- package/evals/verification-loop/eval.yaml +26 -0
- package/evals/verification-loop/tasks/negative-discussion.yaml +16 -0
- package/evals/verification-loop/tasks/positive-pre-ship.yaml +15 -0
- package/evals/verify-before-claim/eval.yaml +26 -0
- package/evals/verify-before-claim/tasks/negative-trigger.yaml +16 -0
- package/evals/verify-before-claim/tasks/positive-trigger.yaml +15 -0
- package/evals/zoom-out/eval.yaml +26 -0
- package/evals/zoom-out/tasks/negative-trivial-edit.yaml +16 -0
- package/evals/zoom-out/tasks/positive-repeated-failure.yaml +15 -0
- package/fable-discipline/SKILL.md +31 -28
- package/fable-discipline/references/discipline-rules.md +22 -0
- package/fable-mode/CONTRACT-FORMAT.md +165 -51
- package/fable-mode/SKILL.md +31 -43
- package/fable-mode/agents/fable-orchestrator.md +159 -92
- package/fable-mode/agents/fable-verifier.md +32 -32
- package/fable-mode/agents/fable-worker-haiku.md +52 -48
- package/fable-mode/agents/fable-worker-sonnet.md +53 -49
- package/fable-mode/execution-guardrails/SKILL.md +43 -87
- package/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/fable-mode/fable-haiku/SKILL.md +36 -48
- package/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/fable-mode/fable-opus/SKILL.md +36 -0
- package/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/fable-mode/model-matrix.json +32 -0
- package/fable-mode/references/execution-phases.md +37 -0
- package/fable-mode/references/model-matrix.md +39 -0
- package/fable-mode/scripts/ensemble-review.js +371 -0
- package/fable-mode/scripts/model-selector.js +260 -0
- package/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/find-skills/SKILL.md +48 -0
- package/find-skills/references/discovery-flow.md +102 -0
- package/find-skills/scripts/use-skill.js +112 -0
- package/git-commit/SKILL.md +28 -45
- package/git-commit/guides/ANGULAR_STYLE.md +60 -60
- package/git-commit/guides/COMMIT_GENERATION.md +23 -23
- package/git-commit/guides/LANGUAGE_DETECTION.md +16 -16
- package/git-commit/guides/MAIN_REPO.md +14 -14
- package/git-commit/guides/SUBMODULES.md +27 -27
- package/git-commit/references/commit-flow.md +30 -0
- package/grill-me/SKILL.md +34 -27
- package/grill-me/references/grilling-playbook.md +38 -0
- package/grill-with-docs/ADR-FORMAT.md +47 -47
- package/grill-with-docs/CONTEXT-FORMAT.md +63 -63
- package/grill-with-docs/SKILL.md +33 -91
- package/grill-with-docs/references/session-playbook.md +99 -0
- package/harness-everything/SKILL.md +27 -125
- package/harness-everything/examples/README.md +9 -9
- package/harness-everything/examples/todo-workflow.md +22 -22
- package/harness-everything/references/router-workflow-plan.md +128 -0
- package/harness-everything/references/skill-registry.md +39 -0
- package/harness-everything/references/triage-and-tiers.md +193 -0
- package/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/harness-everything/scripts/bootstrap.js +212 -140
- package/harness-everything/scripts/ensemble-policy.js +102 -0
- package/harness-everything/scripts/kernel-router.js +212 -0
- package/harness-everything/scripts/router-contract.js +612 -0
- package/harness-everything/scripts/routing-keywords.json +320 -0
- package/harness-everything/scripts/self-heal.js +250 -189
- package/harness-everything/scripts/tier-router.js +459 -219
- package/harness-everything/scripts/verify-gate.js +100 -31
- package/hooks/hooks.json +200 -155
- package/hooks/scripts/action-gate-rules.json +77 -0
- package/hooks/scripts/action-gate.js +528 -0
- package/hooks/scripts/atomic-commit-check.js +68 -68
- package/hooks/scripts/boundary-guard.js +55 -55
- package/hooks/scripts/codex-action-gate-post.js +46 -0
- package/hooks/scripts/codex-permission-request.js +140 -0
- package/hooks/scripts/context-compact.js +115 -114
- package/hooks/scripts/contract-test.js +146 -79
- package/hooks/scripts/depth-guard.js +65 -65
- package/hooks/scripts/lib/fable-contracts.js +121 -0
- package/hooks/scripts/lib/harness-state.js +275 -228
- package/hooks/scripts/lib/platforms/claude.js +97 -61
- package/hooks/scripts/lib/platforms/codex.js +42 -50
- package/hooks/scripts/lib/platforms/continue.js +30 -12
- package/hooks/scripts/lib/platforms/copilot.js +36 -42
- package/hooks/scripts/lib/platforms/cursor.js +23 -5
- package/hooks/scripts/lib/platforms/hermes.js +108 -57
- package/hooks/scripts/lib/platforms/index.js +17 -17
- package/hooks/scripts/lib/platforms/worktrees.js +20 -20
- package/hooks/scripts/openai-session-start.js +81 -0
- package/hooks/scripts/reset-circuit-breaker.js +30 -30
- package/hooks/scripts/rule-of-3-tracker.js +125 -91
- package/hooks/scripts/rule-of-3.js +195 -176
- package/hooks/scripts/state-persist.js +100 -99
- package/hooks/scripts/stop-gate.js +80 -80
- package/hooks/scripts/subagent-scope-guard.js +194 -88
- package/hooks/scripts/subagent-start.js +19 -0
- package/hooks/scripts/subagent-stop.js +6 -0
- package/improve-codebase-architecture/SKILL.md +29 -35
- package/improve-codebase-architecture/guides/DEEPENING.md +37 -37
- package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -123
- package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -44
- package/improve-codebase-architecture/guides/LANGUAGE.md +53 -53
- package/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/improve-codebase-architecture/references/README.md +16 -0
- package/install-cognitive-os/SKILL.md +30 -81
- package/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/multi-agent-workspace/SKILL.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/examples/csharp-monorepo.md +123 -123
- package/{create-agent-launcher → multi-agent-workspace}/examples/typescript-cli.md +126 -126
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-claude.md +77 -77
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-codex.md +146 -146
- package/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-copilot.md +74 -74
- package/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/role-capabilities.md +108 -108
- package/multi-agent-workspace/references/agency-agents.md +31 -0
- package/multi-agent-workspace/references/architecture-guide.md +36 -0
- package/multi-agent-workspace/references/orchestration.md +65 -0
- package/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/multi-agent-workspace/scripts/index_memory.js +104 -0
- package/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/multi-agent-workspace/scripts/scaffold.js +408 -0
- package/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/{create-agent-launcher → multi-agent-workspace}/templates/README.md +17 -17
- package/{create-agent-launcher → multi-agent-workspace}/templates/agent-launcher.md +67 -67
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-csharp.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-polyglot.md +61 -61
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-ts.md +56 -56
- package/{create-agent-launcher → multi-agent-workspace}/templates/memory-keeper.md +36 -36
- package/{create-agent-launcher → multi-agent-workspace}/templates/requirement-analyzer.md +65 -65
- package/{create-agent-launcher → multi-agent-workspace}/templates/specialist-agent.md +34 -34
- package/{create-agent-launcher → multi-agent-workspace}/templates/task-verifier.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/tool-maker.md +32 -32
- package/multi-agent-workspace/workflows/01-init.md +22 -0
- package/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/multi-agent-workspace/workflows/03-generation.md +25 -0
- package/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/opencode-plugin/README.md +137 -0
- package/opencode-plugin/hooks/verify.js +190 -0
- package/opencode-plugin/index.mjs +547 -0
- package/opencode-plugin/plugin.json +9 -0
- package/package.json +63 -22
- package/plugins/harness-everything/.codex-plugin/plugin.json +39 -0
- package/plugins/harness-everything/hooks/hooks.json +197 -0
- package/plugins/harness-everything/hooks/scripts/action-gate-rules.json +77 -0
- package/plugins/harness-everything/hooks/scripts/action-gate.js +528 -0
- package/plugins/harness-everything/hooks/scripts/atomic-commit-check.js +68 -0
- package/plugins/harness-everything/hooks/scripts/boundary-guard.js +55 -0
- package/plugins/harness-everything/hooks/scripts/codex-action-gate-post.js +46 -0
- package/plugins/harness-everything/hooks/scripts/codex-permission-request.js +140 -0
- package/plugins/harness-everything/hooks/scripts/context-compact.js +115 -0
- package/plugins/harness-everything/hooks/scripts/contract-test.js +146 -0
- package/plugins/harness-everything/hooks/scripts/depth-guard.js +65 -0
- package/plugins/harness-everything/hooks/scripts/lib/fable-contracts.js +121 -0
- package/plugins/harness-everything/hooks/scripts/lib/harness-state.js +275 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/claude.js +168 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/codex.js +88 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/continue.js +91 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/copilot.js +85 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/cursor.js +89 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/hermes.js +108 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/index.js +17 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/worktrees.js +20 -0
- package/plugins/harness-everything/hooks/scripts/lib/workspace.js +290 -0
- package/plugins/harness-everything/hooks/scripts/openai-session-start.js +81 -0
- package/plugins/harness-everything/hooks/scripts/reset-circuit-breaker.js +30 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3-tracker.js +125 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3.js +195 -0
- package/plugins/harness-everything/hooks/scripts/state-persist.js +100 -0
- package/plugins/harness-everything/hooks/scripts/stop-gate.js +80 -0
- package/plugins/harness-everything/hooks/scripts/subagent-scope-guard.js +194 -0
- package/plugins/harness-everything/hooks/scripts/subagent-start.js +19 -0
- package/plugins/harness-everything/hooks/scripts/subagent-stop.js +6 -0
- package/plugins/harness-everything/hooks/session-start.js +81 -0
- package/plugins/harness-everything/plugin.json +43 -0
- package/plugins/harness-everything/scripts/lib/advisory-text.js +174 -0
- package/plugins/harness-everything/skills/environment-detection/SKILL.md +41 -0
- package/plugins/harness-everything/skills/environment-detection/references/shell-syntax-rules.md +76 -0
- package/plugins/harness-everything/skills/environment-detection/scripts/preflight.js +103 -0
- package/plugins/harness-everything/skills/eval-harness/SKILL.md +42 -0
- package/plugins/harness-everything/skills/eval-harness/references/scoring-rubric.md +68 -0
- package/plugins/harness-everything/skills/eval-harness/scripts/evaluate.js +79 -0
- package/plugins/harness-everything/skills/fable-discipline/SKILL.md +41 -0
- package/plugins/harness-everything/skills/fable-discipline/references/discipline-rules.md +22 -0
- package/plugins/harness-everything/skills/fable-mode/CONTRACT-FORMAT.md +165 -0
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +159 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-verifier.md +32 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-haiku.md +52 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-sonnet.md +53 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +43 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/model-matrix.json +32 -0
- package/plugins/harness-everything/skills/fable-mode/references/execution-phases.md +37 -0
- package/plugins/harness-everything/skills/fable-mode/references/model-matrix.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/ensemble-review.js +371 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/model-selector.js +260 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/plugins/harness-everything/skills/find-skills/SKILL.md +48 -0
- package/plugins/harness-everything/skills/find-skills/references/discovery-flow.md +102 -0
- package/plugins/harness-everything/skills/find-skills/scripts/use-skill.js +112 -0
- package/plugins/harness-everything/skills/git-commit/SKILL.md +38 -0
- package/plugins/harness-everything/skills/git-commit/guides/ANGULAR_STYLE.md +61 -0
- package/plugins/harness-everything/skills/git-commit/guides/COMMIT_GENERATION.md +24 -0
- package/plugins/harness-everything/skills/git-commit/guides/LANGUAGE_DETECTION.md +16 -0
- package/plugins/harness-everything/skills/git-commit/guides/MAIN_REPO.md +15 -0
- package/plugins/harness-everything/skills/git-commit/guides/SUBMODULES.md +27 -0
- package/plugins/harness-everything/skills/git-commit/references/commit-flow.md +30 -0
- package/plugins/harness-everything/skills/grill-me/SKILL.md +43 -0
- package/plugins/harness-everything/skills/grill-me/references/grilling-playbook.md +38 -0
- package/plugins/harness-everything/skills/grill-with-docs/ADR-FORMAT.md +47 -0
- package/plugins/harness-everything/skills/grill-with-docs/CONTEXT-FORMAT.md +63 -0
- package/plugins/harness-everything/skills/grill-with-docs/SKILL.md +41 -0
- package/plugins/harness-everything/skills/grill-with-docs/references/session-playbook.md +99 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +43 -0
- package/plugins/harness-everything/skills/harness-everything/examples/README.md +9 -0
- package/plugins/harness-everything/skills/harness-everything/examples/todo-workflow.md +22 -0
- package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md +128 -0
- package/plugins/harness-everything/skills/harness-everything/references/skill-registry.md +39 -0
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +193 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/bootstrap.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/ensemble-policy.js +102 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/router-contract.js +612 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/routing-keywords.json +320 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/self-heal.js +250 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +459 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/verify-gate.js +100 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/SKILL.md +38 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/DEEPENING.md +37 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/HTML-REPORT.md +123 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/LANGUAGE.md +53 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/references/README.md +16 -0
- package/plugins/harness-everything/skills/install-cognitive-os/SKILL.md +44 -0
- package/plugins/harness-everything/skills/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/csharp-monorepo.md +123 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/typescript-cli.md +126 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-claude.md +77 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-codex.md +146 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-copilot.md +74 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/role-capabilities.md +108 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/agency-agents.md +31 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/architecture-guide.md +36 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/orchestration.md +65 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/index_memory.js +104 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/scaffold.js +408 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/README.md +17 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/agent-launcher.md +68 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-csharp.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-polyglot.md +62 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-ts.md +57 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/memory-keeper.md +37 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/requirement-analyzer.md +66 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/specialist-agent.md +34 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/task-verifier.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/tool-maker.md +33 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/01-init.md +22 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/03-generation.md +25 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/plugins/harness-everything/skills/repo-docs/SKILL.md +42 -0
- package/plugins/harness-everything/skills/repo-docs/references/process-guide.md +60 -0
- package/plugins/harness-everything/skills/repo-docs/templates/agents-template.md +264 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-agents-template.md +50 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-readme-template.md +83 -0
- package/plugins/harness-everything/skills/repo-docs/templates/manual-recon.md +117 -0
- package/plugins/harness-everything/skills/repo-docs/templates/multi-skills-readme-template.md +72 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-agents-template.md +78 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-readme-template.md +95 -0
- package/plugins/harness-everything/skills/repo-docs/templates/readme-template.md +136 -0
- package/plugins/harness-everything/skills/repo-docs/templates/single-skill-readme-template.md +70 -0
- package/plugins/harness-everything/skills/repo-docs/templates/skills-agents-template.md +60 -0
- package/plugins/harness-everything/skills/rewrite-commits/SKILL.md +37 -0
- package/plugins/harness-everything/skills/rewrite-commits/references/workflow.md +18 -0
- package/plugins/harness-everything/skills/security-review/SKILL.md +47 -0
- package/plugins/harness-everything/skills/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/plugins/harness-everything/skills/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/plugins/harness-everything/skills/security-review/references/security-checklist.md +478 -0
- package/plugins/harness-everything/skills/security-review/scripts/audit-secrets.js +85 -0
- package/plugins/harness-everything/skills/self-evolve/SKILL.md +45 -0
- package/plugins/harness-everything/skills/self-evolve/references/memory-resolution.md +61 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/persist-memory.js +272 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/register-dynamic-skill.js +253 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/self-regression.js +125 -0
- package/plugins/harness-everything/skills/skill-creator/SKILL.md +47 -0
- package/plugins/harness-everything/skills/skill-creator/references/authoring-workflow.md +14 -0
- package/plugins/harness-everything/skills/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-checklist.md +16 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-principles.md +81 -0
- package/plugins/harness-everything/skills/skill-creator/references/testing-workflow.md +27 -0
- package/plugins/harness-everything/skills/skill-style/SKILL.md +42 -0
- package/plugins/harness-everything/skills/skill-style/references/style-guide.md +62 -0
- package/plugins/harness-everything/skills/tdd/SKILL.md +38 -0
- package/plugins/harness-everything/skills/tdd/fixtures/pass-evidence.json +100 -0
- package/plugins/harness-everything/skills/tdd/fixtures/quality-scenarios.json +20 -0
- package/plugins/harness-everything/skills/tdd/guides/deep-modules.md +33 -0
- package/plugins/harness-everything/skills/tdd/guides/interface-design.md +31 -0
- package/plugins/harness-everything/skills/tdd/guides/mocking.md +59 -0
- package/plugins/harness-everything/skills/tdd/guides/refactoring.md +10 -0
- package/plugins/harness-everything/skills/tdd/guides/tests.md +61 -0
- package/plugins/harness-everything/skills/tdd/references/core-discipline.md +51 -0
- package/plugins/harness-everything/skills/tdd/references/integration-testing.md +13 -0
- package/plugins/harness-everything/skills/tdd/references/quality-model.md +68 -0
- package/plugins/harness-everything/skills/tdd/references/unit-testing.md +15 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-evidence.schema.json +211 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-report.schema.json +110 -0
- package/plugins/harness-everything/skills/tdd/scripts/quality-gate.js +475 -0
- package/plugins/harness-everything/skills/to-spec/SKILL.md +40 -0
- package/plugins/harness-everything/skills/to-spec/references/process.md +77 -0
- package/plugins/harness-everything/skills/to-spec/scripts/check-project-docs.js +239 -0
- package/plugins/harness-everything/skills/to-spec/scripts/project-docs-resolver.js +111 -0
- package/plugins/harness-everything/skills/to-spec/templates/cli-reference.md +31 -0
- package/plugins/harness-everything/skills/to-spec/templates/dev-doc.md +27 -0
- package/plugins/harness-everything/skills/to-spec/templates/feature-spec.md +59 -0
- package/plugins/harness-everything/skills/to-spec/templates/schema-doc.md +31 -0
- package/plugins/harness-everything/skills/to-tickets/SKILL.md +42 -0
- package/plugins/harness-everything/skills/to-tickets/references/publishing-and-templates.md +97 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/SKILL.md +42 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/references/execution-guide.md +27 -0
- package/plugins/harness-everything/skills/using-git-worktrees/SKILL.md +49 -0
- package/plugins/harness-everything/skills/using-git-worktrees/references/workflow-details.md +108 -0
- package/plugins/harness-everything/skills/verification-loop/SKILL.md +46 -0
- package/plugins/harness-everything/skills/verification-loop/references/verification-phases.md +116 -0
- package/plugins/harness-everything/skills/verification-loop/templates/verification-report.template.md +15 -0
- package/plugins/harness-everything/skills/verify-before-claim/SKILL.md +43 -0
- package/plugins/harness-everything/skills/verify-before-claim/references/verification-guide.md +73 -0
- package/plugins/harness-everything/skills/zoom-out/SKILL.md +42 -0
- package/plugins/harness-everything/skills/zoom-out/references/circuit-breaker.md +66 -0
- package/plugins/harness-everything/skills/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/references/definition-of-done.md +67 -0
- package/references/performance-checklist.md +153 -0
- package/references/platform-advisory-template.md +58 -0
- package/references/security-checklist.md +205 -0
- package/repo-docs/SKILL.md +34 -37
- package/repo-docs/references/process-guide.md +60 -0
- package/repo-docs/templates/agents-template.md +264 -264
- package/repo-docs/templates/knowledge-agents-template.md +50 -50
- package/repo-docs/templates/knowledge-readme-template.md +83 -83
- package/repo-docs/templates/manual-recon.md +117 -117
- package/repo-docs/templates/multi-skills-readme-template.md +72 -72
- package/repo-docs/templates/product-agents-template.md +78 -78
- package/repo-docs/templates/product-readme-template.md +95 -95
- package/repo-docs/templates/readme-template.md +136 -136
- package/repo-docs/templates/single-skill-readme-template.md +70 -70
- package/repo-docs/templates/skills-agents-template.md +60 -60
- package/rewrite-commits/SKILL.md +22 -17
- package/rewrite-commits/references/workflow.md +18 -0
- package/scripts/build-openai-submission.js +158 -0
- package/scripts/installer.js +621 -506
- package/scripts/lib/advisory-text.js +33 -22
- package/scripts/lib/claude-hooks.js +18 -5
- package/scripts/lib/execution-contract.js +70 -0
- package/scripts/lib/gitignore.js +199 -80
- package/scripts/lib/manifest.js +83 -23
- package/scripts/lib/path-boundary.js +90 -0
- package/scripts/lib/prompts.js +1 -1
- package/scripts/lib/skills.js +191 -51
- package/scripts/lib/workspace.js +269 -7
- package/scripts/sync-openai-plugin.js +91 -0
- package/scripts/sync-release-version.js +212 -0
- package/scripts/verify-install.js +147 -0
- package/security-review/SKILL.md +33 -497
- package/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/security-review/references/security-checklist.md +478 -0
- package/security-review/scripts/audit-secrets.js +85 -0
- package/self-evolve/SKILL.md +36 -45
- package/self-evolve/references/memory-resolution.md +61 -0
- package/self-evolve/scripts/persist-memory.js +272 -40
- package/self-evolve/scripts/register-dynamic-skill.js +253 -0
- package/self-evolve/scripts/self-regression.js +125 -113
- package/skill-creator/SKILL.md +33 -77
- package/skill-creator/references/authoring-workflow.md +14 -0
- package/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/skill-creator/references/quality-checklist.md +16 -0
- package/skill-creator/references/quality-principles.md +81 -81
- package/skill-creator/references/testing-workflow.md +27 -32
- package/skill-style/SKILL.md +33 -39
- package/skill-style/references/style-guide.md +62 -0
- package/submission/openai/README.md +67 -0
- package/submission/openai/listing.json +31 -0
- package/submission/openai/test-cases.json +61 -0
- package/tdd/SKILL.md +30 -45
- package/tdd/fixtures/pass-evidence.json +100 -0
- package/tdd/fixtures/quality-scenarios.json +20 -0
- package/tdd/guides/deep-modules.md +33 -33
- package/tdd/guides/interface-design.md +31 -31
- package/tdd/guides/mocking.md +59 -59
- package/tdd/guides/refactoring.md +10 -10
- package/tdd/guides/tests.md +61 -61
- package/tdd/references/core-discipline.md +51 -0
- package/tdd/references/integration-testing.md +13 -0
- package/tdd/references/quality-model.md +68 -0
- package/tdd/references/unit-testing.md +15 -0
- package/tdd/schemas/quality-evidence.schema.json +211 -0
- package/tdd/schemas/quality-report.schema.json +110 -0
- package/tdd/scripts/quality-gate.js +475 -0
- package/to-spec/SKILL.md +40 -0
- package/to-spec/references/process.md +77 -0
- package/to-spec/scripts/check-project-docs.js +239 -0
- package/to-spec/scripts/project-docs-resolver.js +111 -0
- package/to-spec/templates/cli-reference.md +31 -0
- package/to-spec/templates/dev-doc.md +27 -0
- package/to-spec/templates/feature-spec.md +59 -0
- package/to-spec/templates/schema-doc.md +31 -0
- package/to-tickets/SKILL.md +42 -0
- package/to-tickets/references/publishing-and-templates.md +97 -0
- package/todo-driven-workflow/SKILL.md +28 -40
- package/todo-driven-workflow/references/execution-guide.md +27 -0
- package/using-git-worktrees/SKILL.md +29 -141
- package/using-git-worktrees/references/workflow-details.md +108 -0
- package/verification-loop/SKILL.md +29 -121
- package/verification-loop/references/verification-phases.md +116 -0
- package/verification-loop/templates/verification-report.template.md +15 -0
- package/verify-before-claim/SKILL.md +27 -70
- package/verify-before-claim/references/verification-guide.md +73 -0
- package/zoom-out/SKILL.md +28 -76
- package/zoom-out/references/circuit-breaker.md +66 -0
- package/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/build-multi-agent-system/AGENTS.md +0 -50
- package/build-multi-agent-system/README.md +0 -63
- package/build-multi-agent-system/SKILL.md +0 -68
- package/create-agent-launcher/SKILL.md +0 -48
- package/create-agent-launcher/guidelines/platform-gemini.md +0 -101
- package/create-agent-launcher/workflows/01-init.md +0 -22
- package/create-agent-launcher/workflows/02-analysis.md +0 -26
- package/create-agent-launcher/workflows/03-generation.md +0 -29
- package/create-agent-launcher/workflows/04-launcher.md +0 -20
- package/docs/workflows/build-multi-agent-system.md +0 -62
- package/docs/workflows/create-agent-launcher.md +0 -62
- package/eval-framework/behavioral-test.js +0 -105
- package/eval-framework/cases/case2.json +0 -5
- package/eval-framework/mechanism-test.js +0 -251
- package/harness-everything/scripts/todo-cli.js +0 -157
- package/skill-style/STYLE.md +0 -37
package/docs/architecture.md
CHANGED
|
@@ -1,95 +1,179 @@
|
|
|
1
|
-
# Harness Architecture
|
|
2
|
-
|
|
3
|
-
This document
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## Architectural Overview
|
|
8
|
-
|
|
9
|
-
Harness
|
|
10
|
-
|
|
11
|
-
The
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
---
|
|
88
|
-
|
|
89
|
-
##
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
1
|
+
# Harness Architecture
|
|
2
|
+
|
|
3
|
+
This document describes the internal architecture, lifecycle, and integration boundaries of the Harness behavior layer. The current cross-platform claim boundary is centralized in [platform-capabilities.md](platform-capabilities.md); host sections here must remain consistent with it.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Architectural Overview
|
|
8
|
+
|
|
9
|
+
Harness is a **system supervisor**, not a universal workflow engine. Skills remain independently useful, while runtime mechanisms establish a small cross-cutting contract around software work.
|
|
10
|
+
|
|
11
|
+
The architecture follows a mechanism-first skill mesh with a **minimal kernel**:
|
|
12
|
+
|
|
13
|
+
- classify scope before mutation,
|
|
14
|
+
- require evidence before completion claims,
|
|
15
|
+
- stop repeated same-signature micro-retries and re-plan,
|
|
16
|
+
- leave domain skill selection, ordering, planning style, and delegation to the agent.
|
|
17
|
+
|
|
18
|
+
> **Do not enforce workflow order. Enforce workflow invariants.**
|
|
19
|
+
|
|
20
|
+
This is intentionally different from the older design where Tier 2 implied a fixed TDD/checklist pipeline and Tier 3 implied a mandatory Fable/multi-agent pipeline.
|
|
21
|
+
|
|
22
|
+
```mermaid
|
|
23
|
+
flowchart TD
|
|
24
|
+
subgraph User_Session [User Session]
|
|
25
|
+
U([User Request]) --> Boot[bootstrap.js: Session Start / restore]
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
subgraph Kernel [Harness Kernel]
|
|
29
|
+
Boot --> Preflight[Environment / integration discovery]
|
|
30
|
+
Preflight --> KR[kernel-router.js]
|
|
31
|
+
KR --> TR[tier-router.js<br/>classifier + guide discovery]
|
|
32
|
+
TR --> Inv[Inject minimal invariants]
|
|
33
|
+
Inv --> Choice[Agent chooses useful skills / tactics]
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
subgraph Execution [Agent-Controlled Execution]
|
|
37
|
+
Choice --> Tools[Tool calls / edits / tests]
|
|
38
|
+
Tools --> Evidence{Evidence supports completion?}
|
|
39
|
+
Evidence -- Yes --> Done[Evidence-backed completion]
|
|
40
|
+
Evidence -- No --> Retry[Diagnose / iterate]
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
subgraph Defense [Runtime Boundaries]
|
|
44
|
+
Retry --> CB{Same-signature failure x3?}
|
|
45
|
+
CB -- No --> Tools
|
|
46
|
+
CB -- Yes --> ZO[zoom-out / fresh diagnosis]
|
|
47
|
+
ZO --> Tools
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
subgraph Learning [Optional Learning / State]
|
|
51
|
+
Done --> Record[Record / self-evolve when useful]
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
style Kernel fill:#eceff1,stroke:#37474f,stroke-width:2px,color:#000000
|
|
55
|
+
style Defense fill:#fff9c4,stroke:#fbc02d,stroke-width:2px,color:#000000
|
|
56
|
+
style Learning fill:#e3f2fd,stroke:#1e88e5,stroke-width:2px,color:#000000
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### Router responsibilities
|
|
60
|
+
|
|
61
|
+
`kernel-router.js` is the public runtime entry point. It delegates heuristic classification and dynamic guide/skill discovery to `tier-router.js`, then converts the result into the Harness contract:
|
|
62
|
+
|
|
63
|
+
- recommended Tier + rationale,
|
|
64
|
+
- required invariants,
|
|
65
|
+
- advisory skill suggestions.
|
|
66
|
+
|
|
67
|
+
The kernel deliberately suppresses the old fixed `BASE EXECUTION LOOP` wording so that a host selecting `tdd`, `security-review`, `repo-docs`, or another peer skill cannot accidentally replace or bypass the cross-cutting Harness contract.
|
|
68
|
+
|
|
69
|
+
`harness-everything` remains the public/manual skill entry point for routing, debugging, and re-routing. `install-cognitive-os` remains the explanatory/manual entry point for the Discover → Think → Try → Summarize → Record policy. Automatic correctness must not depend on either one winning host peer-skill selection first.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## Integration Touchpoints
|
|
74
|
+
|
|
75
|
+
Harness aligns to each host's real capabilities. Enforcement strength is platform-specific; shared skill text does not prove mechanism parity.
|
|
76
|
+
|
|
77
|
+
### Self-healing and placement
|
|
78
|
+
|
|
79
|
+
`harness-everything/scripts/self-heal.js` audits supported integration touchpoints and can re-run the idempotent installer to repair missing pieces. If a user intentionally removed an integration, respect that choice.
|
|
80
|
+
|
|
81
|
+
A platform's native files stay where that platform expects them. Harness-owned runtime state converges under the platform's `harness-everything/` state directory, while skills remain in each platform's native skill location. `manifest.json` records installed artifacts so uninstall can remove only Harness-owned files.
|
|
82
|
+
|
|
83
|
+
### Runtime state vs. skill content
|
|
84
|
+
|
|
85
|
+
Runtime state includes hook metadata, circuit-breaker counters, handoff/verification timestamps, and WAL-style session state. Skill content is separate and independently discoverable. This separation lets a domain skill remain useful even when a host does not support Harness runtime hooks.
|
|
86
|
+
|
|
87
|
+
---
|
|
88
|
+
|
|
89
|
+
## Host Adapters
|
|
90
|
+
|
|
91
|
+
### 1. Claude Code — hook-enforced
|
|
92
|
+
|
|
93
|
+
The installer configures native lifecycle hooks and project skills.
|
|
94
|
+
|
|
95
|
+
- `SessionStart`: `bootstrap.js` restores prior state and audits integrations.
|
|
96
|
+
- `UserPromptSubmit`: `kernel-router.js` establishes routing + invariants before peer/domain skill execution.
|
|
97
|
+
- `PreToolUse`: circuit breaker, boundary/depth/context guards, and subagent scope guards can block supported tool calls.
|
|
98
|
+
- `PostToolUse`: records outcomes/state and tracks repeated failures.
|
|
99
|
+
- `Stop`: `stop-gate.js` prevents an edit batch from being claimed complete without successful verification evidence.
|
|
100
|
+
|
|
101
|
+
The prompt hook is intentionally lightweight: it does **not** prescribe TODO/TDD/Fable order. It gives the model the rails and lets the model orchestrate itself.
|
|
102
|
+
|
|
103
|
+
### 2. OpenCode — plugin enforcement, live loading still unverified
|
|
104
|
+
|
|
105
|
+
`opencode-plugin/` maps supported enforcement behavior to OpenCode's real plugin API. Source-level/mechanism tests exist, but live plugin loading remains unverified; do not promote source/mechanism evidence into a live-host enforcement claim until a real OpenCode session artifact exists.
|
|
106
|
+
|
|
107
|
+
### 3. Cursor — skills plus advisory rules
|
|
108
|
+
|
|
109
|
+
The current installer uses `.cursor/skills/` plus `.cursorrules`. Cursor’s documented skill and plugin surfaces are separate from Harness runtime hooks; routing/verification/retry boundaries remain self-directed unless a compatible mechanism is independently verified.
|
|
110
|
+
|
|
111
|
+
### 4. GitHub Copilot agent surfaces — skills plus advisory instructions
|
|
112
|
+
|
|
113
|
+
The current installer uses `.github/skills/` plus `.github/copilot-instructions.md`. GitHub’s Agent Skills paths are documented, but no Harness-specific plugin or live session is verified here.
|
|
114
|
+
|
|
115
|
+
### 5. Codex — local OpenAI plugin plus advisory installer path
|
|
116
|
+
|
|
117
|
+
Codex has two distinct Harness integration surfaces and documentation must keep them separate:
|
|
118
|
+
|
|
119
|
+
- The general `--codex` installer path writes Codex-facing skills/instructions such as `AGENTS.md`; where the host only consumes those instructions, that path is advisory.
|
|
120
|
+
- The packaged local OpenAI plugin under `plugins/harness-everything/` includes `.codex-plugin/plugin.json`, all 26 canonical skills, and lifecycle hooks for `SessionStart`, `UserPromptSubmit`, supported local `PreToolUse` / `PostToolUse`, `SubagentStart` / `SubagentStop`, and `Stop`. Those hooks inject the compact session policy, run the invariant-first kernel, track supported edits and verification, and surface subagent scope changes.
|
|
121
|
+
|
|
122
|
+
The local plugin therefore has **mechanism-tested local enforcement** for the packaged event/tool mappings. This is package evidence, not a live-host claim; the host's hook review/trust flow and a fresh session are still required. Do not infer broader Claude parity for events or tools that are not listed in the package.
|
|
123
|
+
|
|
124
|
+
The public OpenAI **Skills-only** submission is narrower again: it ships reusable skills and referenced assets, not the local `.codex-plugin` lifecycle hooks. Public reviewer/listing claims must describe skill/workflow behavior rather than local hook enforcement. See [openai-plugin.md](openai-plugin.md).
|
|
125
|
+
|
|
126
|
+
### 6. Continue.dev — rules documented, skill discovery unknown
|
|
127
|
+
|
|
128
|
+
The installer writes `.continue/rules/harness.md` with the platform's native rules format and retains `.continue/skills/` as an adapter candidate. The reviewed official Continue documentation does not establish standalone `SKILL.md` discovery, so no skill-loading or hard-gate parity is claimed.
|
|
129
|
+
|
|
130
|
+
### 7. Hermes Agent — trusted skills plus advisory context
|
|
131
|
+
|
|
132
|
+
The installer writes `.hermes.md` for explicit project coverage and can install trusted project skills under `.agents/skills/` plus global skills under `~/.hermes/skills/`. Project skill loading remains subject to Hermes trust; no hard-gate parity is claimed.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## Cognitive OS and Skill Mesh
|
|
137
|
+
|
|
138
|
+
The Cognitive OS is a policy layer, not a parent skill that every domain skill must call.
|
|
139
|
+
|
|
140
|
+
```mermaid
|
|
141
|
+
flowchart LR
|
|
142
|
+
D[Discover] --> T[Think]
|
|
143
|
+
T --> Y[Try]
|
|
144
|
+
Y --> S[Summarize]
|
|
145
|
+
S --> R[Record]
|
|
146
|
+
S -- insufficient evidence --> T
|
|
147
|
+
Y -- same failure x3 --> Z[Zoom Out]
|
|
148
|
+
Z --> T
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
A domain skill may have its own lifecycle — for example RED/GREEN/REFACTOR inside `tdd` — without being forced into a global sequence. The only shared obligations are the kernel invariants.
|
|
152
|
+
|
|
153
|
+
This separation solves the peer-skill routing problem: a strong host may correctly decide that a task primarily needs `tdd` or `security-review`; that choice is allowed. What it cannot silently erase is the routing/evidence/retry contract established by the runtime.
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## Security Model & Data Locality
|
|
158
|
+
|
|
159
|
+
Harness runs with a local-first, zero-trust model:
|
|
160
|
+
|
|
161
|
+
1. **No Harness telemetry:** runtime scripts do not upload project code or state as a Harness service.
|
|
162
|
+
2. **Credential protection:** scripts do not require storing user secrets; interactive credentials remain human-controlled.
|
|
163
|
+
3. **Small hook surface:** hook scripts are local Node.js programs and should stay fast and auditable.
|
|
164
|
+
4. **Explicit enforcement labels:** unsupported/advisory behavior must not be documented as hard enforcement.
|
|
165
|
+
5. **Deterministic regression:** mechanism and routing behavior is tested with executable CI gates rather than inferred from documentation.
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Validation Boundary
|
|
170
|
+
|
|
171
|
+
Static configuration is not enough to claim that a host behaves correctly. Validation is layered:
|
|
172
|
+
|
|
173
|
+
- syntax/reference/manifest checks prove package integrity,
|
|
174
|
+
- `ci/invariant-routing.test.js` proves the kernel contract and documentation invariants,
|
|
175
|
+
- `ci/doc-capability-consistency.test.js` prevents current platform capability claims from drifting apart,
|
|
176
|
+
- mechanism tests prove supported hook/plugin behavior,
|
|
177
|
+
- live host sessions are required before promoting a source/mechanism-tested adapter to live host enforcement.
|
|
178
|
+
|
|
179
|
+
The architecture and its Mermaid diagrams are part of that contract. Runtime changes that alter orchestration must update these documents in the same change.
|
package/docs/audit.md
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Harness System Audit Log
|
|
2
|
+
|
|
3
|
+
Maintainer-facing audit record. If you just want to know whether Harness works and how to verify it yourself, start at [VERIFICATION.md](../VERIFICATION.md) instead — this file records *how the scores below were obtained* and what changed between audit cycles.
|
|
4
|
+
|
|
5
|
+
> **Historical snapshot boundary:** the scorecard below is the 2026-09-01 audit and is intentionally preserved as evidence of what was measured on that date. Platform capabilities have changed since then (including the OpenAI/Codex local plugin work and the OpenCode plugin implementation). Do not treat the 2026-09-01 platform rows as current-state documentation; use [platform-capabilities.md](platform-capabilities.md) for the current matrix.
|
|
6
|
+
|
|
7
|
+
**Last self-audited: 2026-09-01**, by running the actual test suite and the VERIFICATION.md recipes on Windows 11 — not by reading the code and assuming it works.
|
|
8
|
+
|
|
9
|
+
## The 2026-07-23 mis-measurement incident
|
|
10
|
+
|
|
11
|
+
Seven independent AI-model audits (Gemini 3.1 Pro, Gemini 3.5 Flash, GPT/Copilot on Mistral Medium/Small, and others) ran the same week and disagreed sharply on whether core hooks even functioned. Three of them tested `rule-of-3.js`, `boundary-guard.js`, and `state-persist.js` by piping JSON through `echo '...' | node script.js` in Windows Git Bash, which mangles stdin/TTY state and produced false "broken" verdicts. Re-running the identical payloads through Node's own `child_process` stdin API (the technique VERIFICATION.md's recipes now use) showed every one of those mechanisms working exactly as documented — the scores below reflect that corrected, verified state, not the average of the seven reports.
|
|
12
|
+
|
|
13
|
+
This incident is why VERIFICATION.md's recipes were rewritten to be Windows-safe (`node -e` stdin instead of a shell `echo` pipe), and why the §2 mechanism checks were automated as `ci/mechanism-test.js` (`npm run test:mechanism`) — copy-paste recipes that can be mis-executed are not a reliable verification substrate.
|
|
14
|
+
|
|
15
|
+
## Five Core Verification Criteria (per VERIFICATION.md §5a)
|
|
16
|
+
|
|
17
|
+
| Criterion | Score | Verified basis |
|
|
18
|
+
|---|---|---|
|
|
19
|
+
| **Skill Description Completeness** | 10/10 | All 26 skill directories present in the 2026-09-01 snapshot had a complete four-row Skill Contract and the required trigger sections. Four PreToolUse hooks remained internal mechanisms, not standalone skills. |
|
|
20
|
+
| **Routing Accuracy** | 9/10 | The local router passed 34/34 positive prompts across all 26 skills in that snapshot, including the foundation and meta routes added in the audit. Waza verification was not runnable in that Windows checkout; ambiguous/metaphorical generalization remained unmeasured. |
|
|
21
|
+
| **Test Coverage of All Skills** | 9.5/10 | `npm test` included syntax, CLI, routing matrix, positive route coverage, references, behavioral-case structure, Fable contract, and mechanism checks at the time. Waza and live model sessions remained separate CI/on-demand evidence. |
|
|
22
|
+
| **Configuration Balance** | 8.5/10 | **Historical 2026-09-01 result:** hard `exit(2)` blocking hooks were measured on Claude Code and the other then-installed platform paths were evaluated as advisory. This row predates the later OpenAI/Codex local plugin and OpenCode plugin work; see `docs/platform-capabilities.md` for current status. |
|
|
23
|
+
| **Workflow Conformance** | 8.5/10 | Every skill in the snapshot had a matching workflow document, and `ci/skill-routing-check.js` executed every positive route case against the real router. Runtime tool-call sequence conformance remained host-dependent and was not claimed. |
|
|
24
|
+
|
|
25
|
+
## Overall Scorecard
|
|
26
|
+
|
|
27
|
+
| Category | Score | Notes |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| **Architecture** | 9/10 | Per-session state isolation, per-platform state directories, fail-open-by-default hooks — all re-verified directly against `hooks/scripts/lib/harness-state.js` in the 2026-09-01 audit. |
|
|
30
|
+
| **Test Coverage** | 9/10 | Deterministic checks covered syntax, routing, skill references, release-tag catalog, install tree/version drift, behavioral-case structure, and 12 mechanism suites at the time. Live model evaluation remained on-demand and was not represented by a local simulation. |
|
|
31
|
+
| **README Completeness** | 9.5/10 | Catalog counts, layer labels, route coverage, and the local baseline matched the then-current 26-skill tree. |
|
|
32
|
+
| **Maintainability** | 9/10 | The custom cross-session todo CLI and obsolete multi-agent compatibility stubs were absent; route coverage was a deterministic regression gate. |
|
|
33
|
+
| **Skills Design** | 9.5/10 | All 26 skills in the snapshot used the same four-row Skill Contract and exact trigger-section contract; internal hooks remained intentionally non-routable. |
|
|
34
|
+
| **Agent Compatibility** | 9/10 | **Historical 2026-09-01 result:** full hard-mechanism support was measured on Claude Code and advisory fallback on the other installer targets then covered by the audit. This is not the current platform matrix. |
|
|
35
|
+
| **Beginner Friendliness** | 7.5/10 | Quick Start was genuinely 10 seconds, but Tier Routing / Rule of 3 / session-scoped state were load-bearing concepts a newcomer had to absorb before the system's behavior made sense. Not addressed in that audit round. |
|
|
36
|
+
|
|
37
|
+
## Change log by audit cycle
|
|
38
|
+
|
|
39
|
+
### 2026-09-01
|
|
40
|
+
- Fable v3 integration exposed explicit Haiku, Sonnet, and Opus entrypoints, with deterministic model selection and visible inline fallback or blocked escalation in `fable-mode/model-matrix.json` and `fable-mode/scripts/model-selector.js`.
|
|
41
|
+
- The workflow used native host TODO tracking or Markdown checklists. The older shared TODO CLI/state-machine references below are retained only as historical audit evidence; no current runtime, test, or active workflow depends on them.
|
|
42
|
+
- Strict catalog audit: removed obsolete compatibility directories and stale workflow; added the missing `find-skills` workflow.
|
|
43
|
+
- Corrected documentation and runtime references to the per-session `.claude/harness-everything/state/` path, aligned README/registry layer labels, and removed unimplemented workflow claims.
|
|
44
|
+
- Added `ci/skill-routing-check.js`; all 34 positive routing cases across the 26-skill catalog passed locally, and all positive eval descriptions exactly matched their skill frontmatter.
|
|
45
|
+
|
|
46
|
+
### 2026-07-26
|
|
47
|
+
- `verify-gate.js` was no longer a simulated stub: it discovered the nearest `package.json` (scoped to the enclosing git repo), ran its real `lint`/`test` scripts via the detected package manager (npm/pnpm/yarn/bun), and blocked completion on failure. The `.verify-fail.tmp` injection hook was kept for hermetic mechanism tests, and `HARNESS_SKIP_PROJECT_CHECKS=1` guarded against self-recursion in projects whose test suite itself drove the Harness runtime. When no runnable scripts existed it exited 0 but explicitly warned that no mechanical checks ran — the exit code was not evidence.
|
|
48
|
+
- CI added: `.github/workflows/ci.yml` ran `npm test` on push/PR, closing the "local-gate-only" gap called out in the previous cycle.
|
|
49
|
+
- README restructured to be user-facing (problem → quick start → what gets installed/uninstall → platform support); audit narrative moved to this file.
|
|
50
|
+
- `harness-everything/SKILL.md`: tier-router output wording and the Skill Contract were aligned (router output is the default route, overridable with a one-line reason — previously the contract said "MUST follow" while the script itself said "default, not an order"); section numbering fixed (§5 registry now precedes §6).
|
|
51
|
+
|
|
52
|
+
### 2026-07-23
|
|
53
|
+
- VERIFICATION.md §2's mechanism checks automated (`npm run test:mechanism`).
|
|
54
|
+
- VERIFICATION.md's test recipes rewritten to be Windows-safe (`node -e` stdin instead of a shell `echo` pipe).
|
|
55
|
+
- Workflow progress was recorded in the host TODO tracker or a Markdown checklist.
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
# Codex action-gate policy
|
|
2
|
+
|
|
3
|
+
This document records the Codex-specific action-gate contract. It intentionally differs from Claude Code because the two hosts expose different approval semantics.
|
|
4
|
+
|
|
5
|
+
## Current official capability boundary
|
|
6
|
+
|
|
7
|
+
Harness treats the current Codex hooks documentation as the release-behavior contract.
|
|
8
|
+
|
|
9
|
+
### PreToolUse
|
|
10
|
+
|
|
11
|
+
For Codex, Harness may safely block a covered local tool call with a supported deny/block result or exit code 2. Harness must not emit `permissionDecision: "ask"`: Codex currently parses that value but does not support it as an authorization decision, so relying on it would be fail-open.
|
|
12
|
+
|
|
13
|
+
The packaged action gate therefore keeps the existing fail-closed behavior for a destructive rule match:
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
covered local tool
|
|
17
|
+
-> PreToolUse
|
|
18
|
+
-> Harness destructive rule match
|
|
19
|
+
-> exit 2 / block
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
An internal failure in this pre-action path also remains fail-closed.
|
|
23
|
+
|
|
24
|
+
### PermissionRequest
|
|
25
|
+
|
|
26
|
+
Codex exposes a separate `PermissionRequest` lifecycle event when Codex is already going to request approval. A PermissionRequest hook can refine that request, but the event is not a mechanism for manufacturing an approval prompt for an otherwise permitted command.
|
|
27
|
+
|
|
28
|
+
Harness therefore registers `codex-permission-request.js` as an observer only:
|
|
29
|
+
|
|
30
|
+
- it records that Codex entered its native approval flow;
|
|
31
|
+
- it records the Harness rule match when the current rule table recognizes the tool input;
|
|
32
|
+
- it stores a SHA-256 of the exact tool input rather than the raw command/tool arguments;
|
|
33
|
+
- it returns no `allow`, `deny`, or `ask` decision;
|
|
34
|
+
- an observer/audit failure returns no authorization decision, so Codex's already-active native approval flow remains authoritative.
|
|
35
|
+
|
|
36
|
+
The current documented PermissionRequest payload does not provide the same `tool_use_id` correlation used by PreToolUse/PostToolUse. PermissionRequest evidence therefore explicitly records `exactToolCallCorrelation: false` and uses only the turn id plus the payload hash for limited correlation.
|
|
37
|
+
|
|
38
|
+
### PostToolUse
|
|
39
|
+
|
|
40
|
+
Codex uses the documented `PostToolUse` lifecycle for completion evidence, including non-zero Bash outcomes. Harness does not declare a Codex `PostToolUseFailure` hook.
|
|
41
|
+
|
|
42
|
+
`codex-action-gate-post.js` is kept in canonical `hooks/scripts/` and copied into the plugin package by `sync-openai-plugin.js`. This prevents a package sync from deleting the Codex-only adapter.
|
|
43
|
+
|
|
44
|
+
## Native execpolicy prompt rules
|
|
45
|
+
|
|
46
|
+
Codex rules/execpolicy can express native `prompt` decisions for command prefixes. That is a promising native confirmation path, but Harness does not automatically install or bundle those rules yet.
|
|
47
|
+
|
|
48
|
+
Reasons:
|
|
49
|
+
|
|
50
|
+
1. the plugin packaging contract does not establish a plugin-owned execpolicy-rules component that Harness can assume is installed and active;
|
|
51
|
+
2. execpolicy is an approval/sandbox layer and is not proven to cover every destructive operation that may already be allowed in the active sandbox;
|
|
52
|
+
3. rule activation and approval behavior must be verified in the real Codex host before a hard safety guard is weakened.
|
|
53
|
+
|
|
54
|
+
Until live evidence establishes a guaranteed native approval path for a matched action, a Harness destructive match remains a hard PreToolUse block.
|
|
55
|
+
|
|
56
|
+
## Required live verification before native-prompt delegation
|
|
57
|
+
|
|
58
|
+
A future change may delegate selected destructive rules to Codex-native prompting only after preserving evidence for at least:
|
|
59
|
+
|
|
60
|
+
- `approval_policy = "on-request"`;
|
|
61
|
+
- `approval_policy = "never"`;
|
|
62
|
+
- granular approvals with rules enabled;
|
|
63
|
+
- `approvals_reviewer = "user"`;
|
|
64
|
+
- `approvals_reviewer = "auto_review"`;
|
|
65
|
+
- plugin-hook trust/review enabled in the actual host;
|
|
66
|
+
- Windows and one Unix-like environment where practical.
|
|
67
|
+
|
|
68
|
+
The evidence must distinguish:
|
|
69
|
+
|
|
70
|
+
```text
|
|
71
|
+
Harness PreToolUse block
|
|
72
|
+
Codex execpolicy/native prompt
|
|
73
|
+
PermissionRequest defer/allow/deny
|
|
74
|
+
PostToolUse execution result
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Package and mechanism tests must not be promoted to `Live verified` in the compatibility matrix.
|
|
78
|
+
|
|
79
|
+
## Tool coverage
|
|
80
|
+
|
|
81
|
+
Current Harness PreToolUse enforcement remains wired to the tool set explicitly covered by the package (`Bash|apply_patch`). Codex hooks can cover additional local function/MCP tools, but those tools must not be claimed as action-gated until Harness has a safe classifier/rule representation for their structured inputs. Hosted or specialized tool paths may have different hook behavior and remain outside this guarantee unless separately verified.
|
|
82
|
+
|
|
83
|
+
## References
|
|
84
|
+
|
|
85
|
+
- Codex Hooks: https://developers.openai.com/codex/hooks
|
|
86
|
+
- Codex Rules: https://developers.openai.com/codex/rules
|
|
87
|
+
- Codex agent approvals and security: https://developers.openai.com/codex/agent-approvals-security
|
|
88
|
+
- Codex configuration reference: https://developers.openai.com/codex/config-reference
|
|
89
|
+
|
|
90
|
+
Related tracking: #82, #85, #98, #100, #109, #110.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Issue #20 umbrella rollup
|
|
2
|
+
|
|
3
|
+
This rollup turns issue #20's historical checklist into bounded acceptance decisions. The review snapshot is `main` at `ba7853b`, checked on 2026-09-07. The umbrella owns coordination and release evidence; implementation details remain with the linked issue.
|
|
4
|
+
|
|
5
|
+
## Closure status
|
|
6
|
+
|
|
7
|
+
Issue #20 remains open after this PR. This branch adds deterministic parser,
|
|
8
|
+
collision-gate, plugin-runner, and negative-control coverage, but it does not
|
|
9
|
+
provide live opencode session evidence, a paid behavioral rerun, the proposed
|
|
10
|
+
100-case study, or the three skill-specific improvements below. Those gaps are
|
|
11
|
+
recorded explicitly rather than treated as closed by the deterministic gates.
|
|
12
|
+
|
|
13
|
+
## Workstream decisions
|
|
14
|
+
|
|
15
|
+
| Workstream | Evidence in the review snapshot | Acceptance state | Remaining claim boundary |
|
|
16
|
+
| --- | --- | --- | --- |
|
|
17
|
+
| #37 opencode enforcement | The real plugin API is implemented and mechanism coverage exists in `ci/mechanism-2n-opencode-plugin.test.js`; the integrated reflection lifecycle is `d790e7f` on the integration branch. | Integrated, pending final integration gates | No live opencode loading or session smoke evidence was available; do not claim live enforcement. |
|
|
18
|
+
| #52 behavioral transcript parsing | Claude stream fixtures and parser coverage are integrated in `77efb06`; deterministic parser tests are available. | Integrated, pending final integration gates | No new paid behavioral rerun is part of this umbrella. Parse failures and missing tool visibility remain evidence states, not skill failures. |
|
|
19
|
+
| Systematic gate hardening | The local gates now execute real negative controls; YAML frontmatter parsing fails closed in `ci/description-collision.js`, `ci/routing-check.js` skips clearly when waza is absent, and `ci/mechanism-2t-issue-20.test.js` covers CRLF behavioral parsing and plugin validation. | Deterministic hardening integrated | Live platform loading and model behavior remain unverified. |
|
|
20
|
+
| #42 workspace identity and migration | Workspace-keyed state and migration coverage are integrated in `9dd2e3d`; the issue review identified session and migration acceptance as the relevant boundary. | Integrated, pending final integration gates | Do not claim every host/session migration path is live verified until the integration gate and targeted fixtures pass. |
|
|
21
|
+
| #43 multi-agent workspace layout | The reviewed plan makes its root contract depend on #42. No implementation commit was present in the issue #20 review snapshot. | Deferred to its owner | Do not duplicate its layout or runtime work in this umbrella. |
|
|
22
|
+
| #44 progressive disclosure | Documentation and reference checks are integrated in `b33f598`; `ci/disclosure-check.js` and its mechanism coverage are present. | Integrated, pending final integration gates | This records repository documentation evidence; it does not establish behavioral-model effectiveness. |
|
|
23
|
+
| #56 behavioral rubric triage | The review found grader/source-boundary issues, including `command_exit_0` proving the grader command rather than the agent action. | Deferred to its owner | Do not reuse the historical pass-rate claims until the rubric and trace evidence are corrected. |
|
|
24
|
+
| Extended 100-case study | The original issue proposed 100 cases, but the review did not establish a decision threshold or clean attribution between skill text, plugin enforcement, and grader behavior. | Explicitly deferred | No uncontrolled 100-case run is authorized by this rollup. Define paired design, failure taxonomy, and decision rule first. |
|
|
25
|
+
| Five historical baseline failures | The committed 2026-08-27 records show one failed expectation in each of `baseline-debugging` (edge-case consideration), `baseline-performance` (binary-search suggestion), `baseline-security-review` (MD5 identification), `baseline-simple-bugfix` (function understanding), and `verify-before-done` (verification actually run). | Explicitly deferred | These are historical observations, not a new causal investigation or rerun; do not claim they were fixed by this PR. |
|
|
26
|
+
| Skill-specific improvements | No targeted changes were made to `verification-loop`, `zoom-out`, or `security-review` in this branch. | Explicitly deferred | Strengthening verification enforcement, reflection-report detection, and vulnerability detection requires separate design and behavioral evidence. |
|
|
27
|
+
| Release | The tag-triggered full gate and provenance publish workflow already exists. Historical `0.3.6` evidence is recorded in [docs/release-evidence.md](release-evidence.md). | Documentation complete; next release unproven | A prepared `0.3.7-beta` version is not evidence of a tag, npm publication, or GitHub release. |
|
|
28
|
+
|
|
29
|
+
## Focused acceptance checks
|
|
30
|
+
|
|
31
|
+
The integration owner should run these checks after incorporating the issue branches:
|
|
32
|
+
|
|
33
|
+
```text
|
|
34
|
+
npm test
|
|
35
|
+
npm run test:mechanism
|
|
36
|
+
npm run test:consistency
|
|
37
|
+
npm run test:references
|
|
38
|
+
npm run test:release
|
|
39
|
+
npm run test:collision
|
|
40
|
+
npm run test:routing:skills
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
The checks establish deterministic repository and routing health. They do not substitute for live opencode loading, a paid behavioral run, or npm/GitHub release evidence. Those claims require the artifacts named in [RELEASING.md](../RELEASING.md) and [docs/release-evidence.md](release-evidence.md).
|
|
44
|
+
|
|
45
|
+
## Historical evidence boundary
|
|
46
|
+
|
|
47
|
+
The local review artifact `.git/issue-review-20260907/release-evidence.md` records a successful `0.3.6` workflow (`33466676632`) and npm observation. It explicitly says that this is not proof of a `0.3.7-beta` release. This rollup preserves that distinction so an old successful release cannot be mistaken for current publication evidence.
|