harness-everything 0.2.2 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +27 -0
- package/.claude-plugin/plugin.json +51 -0
- package/.gitattributes +1 -0
- package/.github/workflows/behavioral-evals.yml +36 -0
- package/.github/workflows/ci.yml +117 -0
- package/.github/workflows/release.yml +45 -0
- package/.husky/commit-msg +1 -0
- package/.waza.yaml +22 -0
- package/AGENTS.md +60 -2
- package/BENCHMARK_SOP.md +62 -62
- package/CHANGELOG.md +297 -114
- package/CODE_OF_CONDUCT.md +24 -0
- package/CONTRIBUTING.md +47 -0
- package/Dockerfile +18 -18
- package/LICENSE +201 -201
- package/README.md +279 -197
- package/SECURITY.md +33 -0
- package/VERIFICATION.md +394 -347
- package/ab-test-deep-analysis-2026-08-27.md +383 -0
- package/ab-test-results-2026-08-27.md +152 -0
- package/ab-test-statistical-analysis-31cases.md +193 -0
- package/behavioral-evals/README.md +85 -0
- package/behavioral-evals/analyze-plugin-results.js +110 -0
- package/behavioral-evals/cases/baseline-add-feature.yaml +34 -0
- package/behavioral-evals/cases/baseline-api-design.yaml +32 -0
- package/behavioral-evals/cases/baseline-debugging.yaml +31 -0
- package/behavioral-evals/cases/baseline-dependency.yaml +24 -0
- package/behavioral-evals/cases/baseline-documentation.yaml +34 -0
- package/behavioral-evals/cases/baseline-error-handling.yaml +31 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +33 -0
- package/behavioral-evals/cases/baseline-performance-monitoring.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance.yaml +34 -0
- package/behavioral-evals/cases/baseline-refactor-with-tests.yaml +35 -0
- package/behavioral-evals/cases/baseline-refactor.yaml +27 -0
- package/behavioral-evals/cases/baseline-security-review.yaml +28 -0
- package/behavioral-evals/cases/baseline-simple-bugfix.yaml +33 -0
- package/behavioral-evals/cases/baseline-tdd-workflow.yaml +28 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +50 -0
- package/behavioral-evals/cases/pressure-budget-skip-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-bypass-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +37 -0
- package/behavioral-evals/cases/pressure-expert-skip-testing.yaml +31 -0
- package/behavioral-evals/cases/pressure-fatigue-skip.yaml +33 -0
- package/behavioral-evals/cases/pressure-management-skip-all.yaml +28 -0
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +28 -0
- package/behavioral-evals/cases/pressure-skip-docs.yaml +50 -0
- package/behavioral-evals/cases/pressure-skip-error-handling.yaml +35 -0
- package/behavioral-evals/cases/pressure-skip-security.yaml +32 -0
- package/behavioral-evals/cases/pressure-skip-tests.yaml +27 -0
- package/behavioral-evals/cases/pressure-skip-verification.yaml +43 -0
- package/behavioral-evals/cases/pressure-social-skip-review.yaml +28 -0
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +47 -0
- package/behavioral-evals/cases/scope-discipline.yaml +32 -0
- package/behavioral-evals/cases/tier1-no-overplan.yaml +26 -0
- package/behavioral-evals/cases/verify-before-done.yaml +31 -0
- package/behavioral-evals/results/.gitkeep +0 -0
- package/behavioral-evals/results/2026-08-23-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-skip-verification.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-sunk-cost-retry.json +20 -0
- package/behavioral-evals/results/2026-08-23-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-23-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-23-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-api-design.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-debugging.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-dependency.json +16 -0
- package/behavioral-evals/results/2026-08-27-baseline-documentation.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-git-commit.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance-monitoring.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor-with-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-security-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-simple-bugfix.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-tdd-workflow.json +24 -0
- package/behavioral-evals/results/2026-08-27-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-budget-skip-review.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-bypass-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-complexity-skip.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-expert-skip-testing.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-fatigue-skip.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-management-skip-all.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-docs.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-security.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-verification.json +24 -0
- package/behavioral-evals/results/2026-08-27-pressure-social-skip-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-sunk-cost-retry.json +24 -0
- package/behavioral-evals/results/2026-08-27-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-27-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-27-verify-before-done.json +20 -0
- package/behavioral-evals/results/run-20260827-115509.log +287 -0
- package/behavioral-evals/run-with-plugin.js +316 -0
- package/behavioral-evals/run.js +587 -0
- package/benchmarks/README.md +63 -0
- package/benchmarks/ab-test-2026-08-26/results.json +55 -0
- package/benchmarks/results/logs/.gitkeep +0 -0
- package/benchmarks/results/logs/test-a-harness.md +31 -0
- package/benchmarks/results/logs/test-a-vanilla.md +28 -0
- package/benchmarks/results/logs/test-b-micro-loop-harness.md +48 -0
- package/benchmarks/results/logs/test-b-micro-loop-vanilla.md +36 -0
- package/benchmarks/results/logs/test-c-attention-loss-harness.md +40 -0
- package/benchmarks/results/logs/test-c-attention-loss-vanilla.md +63 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-harness.md +24 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-vanilla.md +29 -0
- package/benchmarks/results/logs/test-e-shell-awareness-harness.md +20 -0
- package/benchmarks/results/logs/test-e-shell-awareness-vanilla.md +20 -0
- package/benchmarks/results/logs/test-f-fact-audit-harness.md +16 -0
- package/benchmarks/results/logs/test-f-fact-audit-vanilla.md +16 -0
- package/benchmarks/results/results.json +158 -0
- package/benchmarks/run.js +220 -0
- package/benchmarks/schema.json +44 -0
- package/bin/cli.js +152 -97
- package/ci/ab-test-cases/grill-me-adversarial.yaml +18 -0
- package/ci/ab-test-cases/tdd-test-first.yaml +23 -0
- package/ci/ab-test-cases/verification-loop-gates.yaml +25 -0
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +18 -0
- package/ci/ab-test-harness.js +410 -0
- package/{eval-framework → ci}/cases/case1.json +5 -5
- package/ci/cases/case10-small-feature.json +5 -0
- package/ci/cases/case11-readme-typo-update.json +5 -0
- package/ci/cases/case12-chinese-macro-audit.json +5 -0
- package/{eval-framework → ci}/cases/case2.json +5 -5
- package/{eval-framework → ci}/cases/case3.json +5 -5
- package/{eval-framework → ci}/cases/case4-zh-tier2.json +5 -5
- package/{eval-framework → ci}/cases/case5-zh-tier3.json +5 -5
- package/ci/cases/case6-macro-skill-audit.json +5 -0
- package/ci/cases/case7-macro-reference-audit.json +5 -0
- package/ci/cases/case8-controlled-ab-benchmark.json +5 -0
- package/ci/cases/case9-four-issue-audit.json +5 -0
- package/ci/consistency-check.js +347 -0
- package/ci/description-collision.js +109 -0
- package/ci/fable-mode-test.js +120 -0
- package/ci/fixtures/agency-agents/divisions.json +6 -0
- package/ci/fixtures/agency-agents/engineering/code-reviewer.md +11 -0
- package/ci/fixtures/agency-agents/testing/reality-checker.md +11 -0
- package/ci/fixtures/bad-frontmatter-colon/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-a/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-b/SKILL.md +16 -0
- package/ci/mechanism-2a-rule-of-3.test.js +46 -0
- package/ci/mechanism-2b-boundary-guard.test.js +16 -0
- package/ci/mechanism-2c-state-persist.test.js +29 -0
- package/ci/mechanism-2d-fact-audit.test.js +18 -0
- package/ci/mechanism-2e-scope-guard.test.js +24 -0
- package/ci/mechanism-2f-stop-gate.test.js +23 -0
- package/ci/mechanism-2g-platform-ignore.test.js +88 -0
- package/ci/mechanism-2h-installer-manifest.test.js +147 -0
- package/ci/mechanism-2i-crlf-frontmatter.test.js +145 -0
- package/ci/mechanism-2i-install-verification.test.js +34 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +20 -0
- package/ci/mechanism-2k-reference-check.test.js +20 -0
- package/ci/mechanism-2l-multi-agent-workspace.test.js +105 -0
- package/ci/mechanism-test.js +71 -0
- package/ci/negative-controls.js +84 -0
- package/ci/reference-check.js +112 -0
- package/ci/release-consistency-check.js +123 -0
- package/ci/routing-check.js +90 -0
- package/{eval-framework → ci}/runner.js +56 -56
- package/ci/skill-routing-check.js +98 -0
- package/ci/test-helper.js +104 -0
- package/ci/test-negative-controls.js +148 -0
- package/commitlint.config.js +25 -0
- package/docker-verify.sh +266 -266
- package/docs/architecture.md +99 -95
- package/docs/audit.md +53 -0
- package/docs/mechanism-first-skill-mesh.md +163 -0
- package/docs/philosophy.md +47 -45
- package/docs/reflection.md +48 -34
- package/docs/routing.md +58 -45
- package/docs/troubleshooting.md +284 -0
- package/docs/workflows/environment-detection.md +10 -10
- package/docs/workflows/eval-harness.md +13 -6
- package/docs/workflows/fable-discipline.md +9 -9
- package/docs/workflows/fable-mode.md +29 -32
- package/docs/workflows/find-skills.md +36 -0
- package/docs/workflows/git-commit.md +18 -9
- package/docs/workflows/grill-me.md +11 -11
- package/docs/workflows/grill-with-docs.md +14 -7
- package/docs/workflows/harness-everything.md +95 -92
- package/docs/workflows/improve-codebase-architecture.md +15 -7
- package/docs/workflows/install-cognitive-os.md +60 -60
- package/docs/workflows/multi-agent-workspace.md +42 -0
- package/docs/workflows/repo-docs.md +12 -7
- package/docs/workflows/security-review.md +26 -7
- package/docs/workflows/self-evolve.md +27 -44
- package/docs/workflows/skill-creator.md +80 -75
- package/docs/workflows/skill-style.md +64 -64
- package/docs/workflows/tdd.md +42 -39
- package/docs/workflows/to-spec.md +83 -0
- package/docs/workflows/to-tickets.md +80 -0
- package/docs/workflows/todo-driven-workflow.md +34 -36
- package/docs/workflows/using-git-worktrees.md +12 -8
- package/docs/workflows/verification-loop.md +16 -14
- package/docs/workflows/verify-before-claim.md +12 -8
- package/docs/workflows/zoom-out.md +22 -10
- package/environment-detection/SKILL.md +25 -61
- package/environment-detection/references/shell-syntax-rules.md +76 -0
- package/environment-detection/scripts/preflight.js +103 -93
- package/eval-framework/fixtures/bad-frontmatter-colon/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-a/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-b/SKILL.md +14 -0
- package/eval-framework/fixtures/missing-delimiter/SKILL.md +13 -0
- package/eval-framework/fixtures/over-budget-skill/SKILL.md +36 -0
- package/eval-harness/SKILL.md +34 -51
- package/eval-harness/references/scoring-rubric.md +68 -0
- package/eval-harness/scripts/evaluate.js +77 -61
- package/evals/environment-detection/eval.yaml +26 -0
- package/evals/environment-detection/tasks/negative-trigger.yaml +16 -0
- package/evals/environment-detection/tasks/positive-trigger.yaml +15 -0
- package/evals/eval-harness/eval.yaml +26 -0
- package/evals/eval-harness/tasks/negative-trigger.yaml +16 -0
- package/evals/eval-harness/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-discipline/eval.yaml +26 -0
- package/evals/fable-discipline/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-discipline/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-mode/eval.yaml +26 -0
- package/evals/fable-mode/tasks/negative-tier2-no-model.yaml +18 -0
- package/evals/fable-mode/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-mode/tasks/positive-haiku-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-opus-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-sonnet-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-trigger.yaml +15 -0
- package/evals/find-skills/eval.yaml +26 -0
- package/evals/find-skills/tasks/negative-trigger.yaml +16 -0
- package/evals/find-skills/tasks/positive-trigger.yaml +15 -0
- package/evals/git-commit/eval.yaml +26 -0
- package/evals/git-commit/tasks/negative-trigger.yaml +16 -0
- package/evals/git-commit/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-me/eval.yaml +26 -0
- package/evals/grill-me/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-me/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-with-docs/eval.yaml +26 -0
- package/evals/grill-with-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-with-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/harness-everything/eval.yaml +26 -0
- package/evals/harness-everything/tasks/negative-non-software.yaml +16 -0
- package/evals/harness-everything/tasks/negative-skill-indicated.yaml +15 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier2.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier3.yaml +16 -0
- package/evals/improve-codebase-architecture/eval.yaml +26 -0
- package/evals/improve-codebase-architecture/tasks/negative-trigger.yaml +16 -0
- package/evals/improve-codebase-architecture/tasks/positive-trigger.yaml +15 -0
- package/evals/install-cognitive-os/eval.yaml +26 -0
- package/evals/install-cognitive-os/tasks/negative-trigger.yaml +16 -0
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/eval.yaml +26 -0
- package/evals/multi-agent-workspace/tasks/negative-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +15 -0
- package/evals/repo-docs/eval.yaml +26 -0
- package/evals/repo-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/repo-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/rewrite-commits/eval.yaml +26 -0
- package/evals/rewrite-commits/tasks/negative-trigger.yaml +16 -0
- package/evals/rewrite-commits/tasks/positive-trigger.yaml +15 -0
- package/evals/security-review/eval.yaml +26 -0
- package/evals/security-review/tasks/negative-trigger.yaml +16 -0
- package/evals/security-review/tasks/positive-trigger.yaml +15 -0
- package/evals/self-evolve/eval.yaml +26 -0
- package/evals/self-evolve/tasks/negative-trigger.yaml +16 -0
- package/evals/self-evolve/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-creator/eval.yaml +26 -0
- package/evals/skill-creator/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-creator/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-style/eval.yaml +26 -0
- package/evals/skill-style/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-style/tasks/positive-trigger.yaml +15 -0
- package/evals/tdd/eval.yaml +26 -0
- package/evals/tdd/tasks/negative-trivia.yaml +16 -0
- package/evals/tdd/tasks/positive-bugfix.yaml +15 -0
- package/evals/tdd/tasks/positive-feature.yaml +16 -0
- package/evals/tdd/tasks/positive-refactor.yaml +15 -0
- package/evals/to-spec/eval.yaml +26 -0
- package/evals/to-spec/tasks/negative-trigger.yaml +16 -0
- package/evals/to-spec/tasks/positive-trigger.yaml +15 -0
- package/evals/to-tickets/eval.yaml +26 -0
- package/evals/to-tickets/tasks/negative-trigger.yaml +15 -0
- package/evals/to-tickets/tasks/positive-trigger.yaml +15 -0
- package/evals/todo-driven-workflow/eval.yaml +26 -0
- package/evals/todo-driven-workflow/tasks/negative-single-edit.yaml +16 -0
- package/evals/todo-driven-workflow/tasks/positive-multi-step.yaml +15 -0
- package/evals/todo-driven-workflow/tasks/positive-refactor-flow.yaml +15 -0
- package/evals/using-git-worktrees/eval.yaml +26 -0
- package/evals/using-git-worktrees/tasks/negative-trigger.yaml +15 -0
- package/evals/using-git-worktrees/tasks/positive-trigger.yaml +15 -0
- package/evals/verification-loop/eval.yaml +26 -0
- package/evals/verification-loop/tasks/negative-discussion.yaml +16 -0
- package/evals/verification-loop/tasks/positive-pre-ship.yaml +15 -0
- package/evals/verify-before-claim/eval.yaml +26 -0
- package/evals/verify-before-claim/tasks/negative-trigger.yaml +16 -0
- package/evals/verify-before-claim/tasks/positive-trigger.yaml +15 -0
- package/evals/zoom-out/eval.yaml +26 -0
- package/evals/zoom-out/tasks/negative-trivial-edit.yaml +16 -0
- package/evals/zoom-out/tasks/positive-repeated-failure.yaml +15 -0
- package/fable-discipline/SKILL.md +31 -28
- package/fable-discipline/references/discipline-rules.md +22 -0
- package/fable-mode/CONTRACT-FORMAT.md +51 -51
- package/fable-mode/SKILL.md +31 -43
- package/fable-mode/agents/fable-orchestrator.md +99 -92
- package/fable-mode/agents/fable-verifier.md +32 -32
- package/fable-mode/agents/fable-worker-haiku.md +52 -48
- package/fable-mode/agents/fable-worker-sonnet.md +53 -49
- package/fable-mode/execution-guardrails/SKILL.md +43 -87
- package/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/fable-mode/fable-haiku/SKILL.md +36 -48
- package/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/fable-mode/fable-opus/SKILL.md +36 -0
- package/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/fable-mode/model-matrix.json +32 -0
- package/fable-mode/references/execution-phases.md +37 -0
- package/fable-mode/references/model-matrix.md +39 -0
- package/fable-mode/scripts/model-selector.js +129 -0
- package/find-skills/SKILL.md +48 -0
- package/find-skills/references/discovery-flow.md +102 -0
- package/find-skills/scripts/use-skill.js +106 -0
- package/git-commit/SKILL.md +35 -45
- package/git-commit/guides/ANGULAR_STYLE.md +60 -60
- package/git-commit/guides/COMMIT_GENERATION.md +23 -23
- package/git-commit/guides/LANGUAGE_DETECTION.md +16 -16
- package/git-commit/guides/MAIN_REPO.md +14 -14
- package/git-commit/guides/SUBMODULES.md +27 -27
- package/git-commit/references/commit-flow.md +30 -0
- package/grill-me/SKILL.md +34 -27
- package/grill-me/references/grilling-playbook.md +38 -0
- package/grill-with-docs/ADR-FORMAT.md +47 -47
- package/grill-with-docs/CONTEXT-FORMAT.md +63 -63
- package/grill-with-docs/SKILL.md +33 -91
- package/grill-with-docs/references/session-playbook.md +97 -0
- package/harness-everything/SKILL.md +29 -128
- package/harness-everything/examples/README.md +9 -9
- package/harness-everything/examples/todo-workflow.md +22 -22
- package/harness-everything/references/skill-registry.md +39 -0
- package/harness-everything/references/triage-and-tiers.md +116 -0
- package/harness-everything/scripts/bootstrap.js +203 -140
- package/harness-everything/scripts/routing-keywords.json +320 -0
- package/harness-everything/scripts/self-heal.js +241 -189
- package/harness-everything/scripts/tier-router.js +334 -219
- package/harness-everything/scripts/verify-gate.js +100 -31
- package/hooks/hooks.json +155 -155
- package/hooks/scripts/atomic-commit-check.js +68 -68
- package/hooks/scripts/boundary-guard.js +55 -55
- package/hooks/scripts/context-compact.js +114 -114
- package/hooks/scripts/contract-test.js +79 -79
- package/hooks/scripts/depth-guard.js +65 -65
- package/hooks/scripts/lib/harness-state.js +139 -228
- package/hooks/scripts/lib/platforms/claude.js +72 -19
- package/hooks/scripts/lib/platforms/codex.js +24 -6
- package/hooks/scripts/lib/platforms/continue.js +23 -5
- package/hooks/scripts/lib/platforms/copilot.js +23 -5
- package/hooks/scripts/lib/platforms/cursor.js +23 -5
- package/hooks/scripts/lib/platforms/hermes.js +57 -57
- package/hooks/scripts/lib/platforms/index.js +17 -17
- package/hooks/scripts/lib/platforms/worktrees.js +20 -20
- package/hooks/scripts/reset-circuit-breaker.js +30 -30
- package/hooks/scripts/rule-of-3-tracker.js +124 -91
- package/hooks/scripts/rule-of-3.js +187 -176
- package/hooks/scripts/state-persist.js +99 -99
- package/hooks/scripts/stop-gate.js +80 -80
- package/hooks/scripts/subagent-scope-guard.js +88 -88
- package/improve-codebase-architecture/SKILL.md +31 -35
- package/improve-codebase-architecture/guides/DEEPENING.md +37 -37
- package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -123
- package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -44
- package/improve-codebase-architecture/guides/LANGUAGE.md +53 -53
- package/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/install-cognitive-os/SKILL.md +23 -80
- package/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/multi-agent-workspace/SKILL.md +37 -0
- package/{create-agent-launcher → multi-agent-workspace}/examples/csharp-monorepo.md +123 -123
- package/{create-agent-launcher → multi-agent-workspace}/examples/typescript-cli.md +126 -126
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-claude.md +77 -77
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-codex.md +146 -146
- package/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-copilot.md +74 -74
- package/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/role-capabilities.md +108 -108
- package/multi-agent-workspace/references/agency-agents.md +31 -0
- package/multi-agent-workspace/references/architecture-guide.md +32 -0
- package/multi-agent-workspace/references/orchestration.md +48 -0
- package/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/multi-agent-workspace/scripts/index_memory.js +51 -0
- package/multi-agent-workspace/scripts/scaffold.js +253 -0
- package/{create-agent-launcher → multi-agent-workspace}/templates/README.md +17 -17
- package/{create-agent-launcher → multi-agent-workspace}/templates/agent-launcher.md +67 -67
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-csharp.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-polyglot.md +61 -61
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-ts.md +56 -56
- package/{create-agent-launcher → multi-agent-workspace}/templates/memory-keeper.md +36 -36
- package/{create-agent-launcher → multi-agent-workspace}/templates/requirement-analyzer.md +65 -65
- package/{create-agent-launcher → multi-agent-workspace}/templates/specialist-agent.md +34 -34
- package/{create-agent-launcher → multi-agent-workspace}/templates/task-verifier.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/tool-maker.md +32 -32
- package/multi-agent-workspace/workflows/01-init.md +22 -0
- package/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/multi-agent-workspace/workflows/03-generation.md +27 -0
- package/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/opencode-plugin/README.md +94 -0
- package/opencode-plugin/hooks/circuit-breaker.js +121 -0
- package/opencode-plugin/hooks/compliance.js +128 -0
- package/opencode-plugin/hooks/post-edit.js +62 -0
- package/opencode-plugin/hooks/pre-complete.js +65 -0
- package/opencode-plugin/hooks/verify.js +140 -0
- package/opencode-plugin/plugin.json +17 -0
- package/opencode-plugin/test-hook-firing.js +198 -0
- package/package.json +50 -22
- package/references/definition-of-done.md +67 -0
- package/references/performance-checklist.md +153 -0
- package/references/platform-advisory-template.md +58 -0
- package/references/security-checklist.md +205 -0
- package/repo-docs/SKILL.md +38 -37
- package/repo-docs/references/process-guide.md +60 -0
- package/repo-docs/templates/agents-template.md +264 -264
- package/repo-docs/templates/knowledge-agents-template.md +50 -50
- package/repo-docs/templates/knowledge-readme-template.md +83 -83
- package/repo-docs/templates/manual-recon.md +117 -117
- package/repo-docs/templates/multi-skills-readme-template.md +72 -72
- package/repo-docs/templates/product-agents-template.md +78 -78
- package/repo-docs/templates/product-readme-template.md +95 -95
- package/repo-docs/templates/readme-template.md +136 -136
- package/repo-docs/templates/single-skill-readme-template.md +70 -70
- package/repo-docs/templates/skills-agents-template.md +60 -60
- package/rewrite-commits/SKILL.md +22 -17
- package/rewrite-commits/references/workflow.md +18 -0
- package/scripts/installer.js +546 -506
- package/scripts/lib/advisory-text.js +8 -4
- package/scripts/lib/claude-hooks.js +18 -5
- package/scripts/lib/gitignore.js +199 -80
- package/scripts/lib/manifest.js +68 -4
- package/scripts/lib/prompts.js +1 -1
- package/scripts/lib/skills.js +87 -1
- package/scripts/verify-install.js +147 -0
- package/security-review/SKILL.md +35 -496
- package/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/security-review/references/security-checklist.md +478 -0
- package/security-review/scripts/audit-secrets.js +84 -0
- package/self-evolve/SKILL.md +38 -45
- package/self-evolve/references/memory-resolution.md +58 -0
- package/self-evolve/scripts/persist-memory.js +139 -40
- package/self-evolve/scripts/register-dynamic-skill.js +251 -0
- package/self-evolve/scripts/self-regression.js +110 -113
- package/skill-creator/SKILL.md +34 -76
- package/skill-creator/references/authoring-workflow.md +14 -0
- package/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/skill-creator/references/quality-checklist.md +15 -0
- package/skill-creator/references/quality-principles.md +81 -81
- package/skill-creator/references/testing-workflow.md +27 -32
- package/skill-style/SKILL.md +33 -39
- package/skill-style/references/style-guide.md +36 -0
- package/tdd/SKILL.md +34 -43
- package/tdd/guides/deep-modules.md +33 -33
- package/tdd/guides/interface-design.md +31 -31
- package/tdd/guides/mocking.md +59 -59
- package/tdd/guides/refactoring.md +10 -10
- package/tdd/guides/tests.md +61 -61
- package/tdd/references/core-discipline.md +49 -0
- package/to-spec/SKILL.md +42 -0
- package/to-spec/references/process.md +77 -0
- package/to-spec/scripts/check-project-docs.js +259 -0
- package/to-spec/templates/cli-reference.md +31 -0
- package/to-spec/templates/dev-doc.md +27 -0
- package/to-spec/templates/feature-spec.md +59 -0
- package/to-spec/templates/schema-doc.md +31 -0
- package/to-tickets/SKILL.md +42 -0
- package/to-tickets/references/publishing-and-templates.md +97 -0
- package/todo-driven-workflow/SKILL.md +27 -41
- package/todo-driven-workflow/references/execution-guide.md +27 -0
- package/using-git-worktrees/SKILL.md +29 -141
- package/using-git-worktrees/references/workflow-details.md +108 -0
- package/verification-loop/SKILL.md +29 -121
- package/verification-loop/references/verification-phases.md +116 -0
- package/verification-loop/templates/verification-report.template.md +15 -0
- package/verify-before-claim/SKILL.md +27 -70
- package/verify-before-claim/references/verification-guide.md +73 -0
- package/zoom-out/SKILL.md +28 -76
- package/zoom-out/references/circuit-breaker.md +66 -0
- package/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/build-multi-agent-system/AGENTS.md +0 -50
- package/build-multi-agent-system/README.md +0 -63
- package/build-multi-agent-system/SKILL.md +0 -68
- package/create-agent-launcher/SKILL.md +0 -48
- package/create-agent-launcher/guidelines/platform-gemini.md +0 -101
- package/create-agent-launcher/workflows/01-init.md +0 -22
- package/create-agent-launcher/workflows/02-analysis.md +0 -26
- package/create-agent-launcher/workflows/03-generation.md +0 -29
- package/create-agent-launcher/workflows/04-launcher.md +0 -20
- package/docs/workflows/build-multi-agent-system.md +0 -62
- package/docs/workflows/create-agent-launcher.md +0 -62
- package/eval-framework/behavioral-test.js +0 -105
- package/eval-framework/mechanism-test.js +0 -251
- package/harness-everything/scripts/todo-cli.js +0 -157
- package/skill-style/STYLE.md +0 -37
package/README.md
CHANGED
|
@@ -1,197 +1,279 @@
|
|
|
1
|
-
# Harness (Behavior Layer for AI Coding Agents)
|
|
2
|
-
|
|
3
|
-
[](LICENSE)
|
|
4
|
-
|
|
5
|
-
Harness is a lightweight, local behavior and orchestration runtime that wraps around your AI development sessions (Claude Code, Cursor, Copilot Chat, Codex, Continue.dev, Hermes Agent). It provides reactive hooks, routing boundaries, and circuit breakers designed to prevent infinite trial-and-error loops, costly over-engineering, and "lost-in-the-middle" context drift.
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## The Problem
|
|
10
|
-
|
|
11
|
-
AI coding agents are highly capable, but they struggle with self-regulation, environment awareness, and attention limits:
|
|
12
|
-
1. **The Infinite Retry Loop:** When an agent encounters a subtle compilation or test failure, its default behavior is to make micro-adjustments repeatedly (tweak and run, tweak and run) until it exhausts your token budget.
|
|
13
|
-
2. **Environment Blindness:** Agents often assume standard Unix environments, hallucinating shell commands and paths when running on Windows, PowerShell, or sandboxed environments.
|
|
14
|
-
3. **Lost-in-the-Middle Bloat:** As sessions grow, agents aggressively read too many large files or generate massive console logs, causing severe context degradation and reasoning hallucinations.
|
|
15
|
-
|
|
16
|
-
---
|
|
17
|
-
|
|
18
|
-
## Why Harness?
|
|
19
|
-
|
|
20
|
-
Harness acts as an automated system supervisor. It remains completely silent and out of the way, intervening only when execution boundaries are violated or failures are detected.
|
|
21
|
-
|
|
22
|
-
### Comparison: Prompt vs. Skill vs. Harness
|
|
23
|
-
|
|
24
|
-
| Dimension | Prompt-Only (Custom Instructions) | Skill-Only (Task Guides) | Harness (Behavior Layer) |
|
|
25
|
-
|---|---|---|---|
|
|
26
|
-
| **Activation** | Always loaded (wastes prompt space) | Loaded on demand (requires manual trigger) | Reacts dynamically via native lifecycle hooks |
|
|
27
|
-
| **Fail-Safe** | No protection (model keeps retrying) | No protection (loops until token limit) | **Circuit Breaker:** Halts execution after 3 failures and alerts human |
|
|
28
|
-
| **Context Aware** | High risk of lost-in-the-middle bloat | Manages scope manually | **Bloat Shield:** Proactively audits diff sizes and logs warning alerts |
|
|
29
|
-
| **System Audit** | Blindly assumes shell syntax | Requires manual shell check | **Preflight:** Proactively detects Windows/Unix paths, shell type, and package manager |
|
|
30
|
-
| **Memory** | Resets on every new chat session | Static text rules | **Continuous Persistence:** Writes Write-Ahead Logs (WAL) for session recovery and immunizes workspace rules |
|
|
31
|
-
|
|
32
|
-
The "Harness" column above is Claude Code's behavior. On Cursor, Copilot, Codex, Continue.dev, and Hermes Agent — platforms with no hook/exit-code execution mechanism — Harness can only inject advisory text, which lands in the **Prompt-Only** column instead. See [Supported AI IDEs & Tools](#supported-ai-ides--tools) below.
|
|
33
|
-
|
|
34
|
-
### When should I use Harness?
|
|
35
|
-
* You regularly use agentic coding tools (like Claude Code, Cursor, or Copilot) on medium-to-large codebases.
|
|
36
|
-
* You develop on Windows or in mixed shells (Git Bash, WSL, PowerShell) where agents frequently get shell syntax wrong.
|
|
37
|
-
* You want automated test-driven development (TDD) enforcement and safety guards to save token budgets.
|
|
38
|
-
|
|
39
|
-
### When should I NOT use Harness?
|
|
40
|
-
* You only use chat interfaces for general questions without letting the AI run local commands or modify files.
|
|
41
|
-
* Your project has no test suite, or you prefer unconstrained, free-form agent generation.
|
|
42
|
-
|
|
43
|
-
---
|
|
44
|
-
|
|
45
|
-
## ⚡ Quick Start (Get Protected in 10s)
|
|
46
|
-
|
|
47
|
-
Harness integrates directly into your workspace. There is no heavy daemon, no paid external APIs, and zero configuration required.
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
#
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
###
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
---
|
|
156
|
-
|
|
157
|
-
##
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
|
178
|
-
|
|
179
|
-
| **
|
|
180
|
-
| **
|
|
181
|
-
| **
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
1
|
+
# Harness (Behavior Layer for AI Coding Agents)
|
|
2
|
+
|
|
3
|
+
[](LICENSE)
|
|
4
|
+
|
|
5
|
+
Harness is a lightweight, local behavior and orchestration runtime that wraps around your AI development sessions (Claude Code, Cursor, Copilot Chat, Codex, Continue.dev, Hermes Agent). It provides reactive hooks, routing boundaries, and circuit breakers designed to prevent infinite trial-and-error loops, costly over-engineering, and "lost-in-the-middle" context drift.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## The Problem
|
|
10
|
+
|
|
11
|
+
AI coding agents are highly capable, but they struggle with self-regulation, environment awareness, and attention limits:
|
|
12
|
+
1. **The Infinite Retry Loop:** When an agent encounters a subtle compilation or test failure, its default behavior is to make micro-adjustments repeatedly (tweak and run, tweak and run) until it exhausts your token budget.
|
|
13
|
+
2. **Environment Blindness:** Agents often assume standard Unix environments, hallucinating shell commands and paths when running on Windows, PowerShell, or sandboxed environments.
|
|
14
|
+
3. **Lost-in-the-Middle Bloat:** As sessions grow, agents aggressively read too many large files or generate massive console logs, causing severe context degradation and reasoning hallucinations.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Why Harness?
|
|
19
|
+
|
|
20
|
+
Harness acts as an automated system supervisor. It remains completely silent and out of the way, intervening only when execution boundaries are violated or failures are detected.
|
|
21
|
+
|
|
22
|
+
### Comparison: Prompt vs. Skill vs. Harness
|
|
23
|
+
|
|
24
|
+
| Dimension | Prompt-Only (Custom Instructions) | Skill-Only (Task Guides) | Harness (Behavior Layer) |
|
|
25
|
+
|---|---|---|---|
|
|
26
|
+
| **Activation** | Always loaded (wastes prompt space) | Loaded on demand (requires manual trigger) | Reacts dynamically via native lifecycle hooks |
|
|
27
|
+
| **Fail-Safe** | No protection (model keeps retrying) | No protection (loops until token limit) | **Circuit Breaker:** Halts execution after 3 failures and alerts human |
|
|
28
|
+
| **Context Aware** | High risk of lost-in-the-middle bloat | Manages scope manually | **Bloat Shield:** Proactively audits diff sizes and logs warning alerts |
|
|
29
|
+
| **System Audit** | Blindly assumes shell syntax | Requires manual shell check | **Preflight:** Proactively detects Windows/Unix paths, shell type, and package manager |
|
|
30
|
+
| **Memory** | Resets on every new chat session | Static text rules | **Continuous Persistence:** Writes Write-Ahead Logs (WAL) for session recovery and immunizes workspace rules |
|
|
31
|
+
|
|
32
|
+
The "Harness" column above is Claude Code's behavior. On Cursor, Copilot, Codex, Continue.dev, and Hermes Agent — platforms with no hook/exit-code execution mechanism — Harness can only inject advisory text, which lands in the **Prompt-Only** column instead. See [Supported AI IDEs & Tools](#supported-ai-ides--tools) below.
|
|
33
|
+
|
|
34
|
+
### When should I use Harness?
|
|
35
|
+
* You regularly use agentic coding tools (like Claude Code, Cursor, or Copilot) on medium-to-large codebases.
|
|
36
|
+
* You develop on Windows or in mixed shells (Git Bash, WSL, PowerShell) where agents frequently get shell syntax wrong.
|
|
37
|
+
* You want automated test-driven development (TDD) enforcement and safety guards to save token budgets.
|
|
38
|
+
|
|
39
|
+
### When should I NOT use Harness?
|
|
40
|
+
* You only use chat interfaces for general questions without letting the AI run local commands or modify files.
|
|
41
|
+
* Your project has no test suite, or you prefer unconstrained, free-form agent generation.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## ⚡ Quick Start (Get Protected in 10s)
|
|
46
|
+
|
|
47
|
+
Harness integrates directly into your workspace. There is no heavy daemon, no paid external APIs, and zero configuration required.
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
# Option A: Claude Code plugin (marketplace manifest included)
|
|
51
|
+
# /plugin marketplace add dyphn1/Harness-everything
|
|
52
|
+
# /plugin install harness-everything
|
|
53
|
+
|
|
54
|
+
# Option B: install Harness hooks and skills into your current workspace
|
|
55
|
+
npx github:dyphn1/Harness-everything install
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### Expected Behavior After Installation:
|
|
59
|
+
1. **Hook Registration:** Harness registers native hooks (e.g., inside `.claude/settings.json` for Claude Code) to intercept session starts and tool use.
|
|
60
|
+
2. **Preflight Audit:** At session startup, a lightweight preflight script runs, printing a diagnostic environment block that tells the agent your exact OS, active shell, and package manager.
|
|
61
|
+
3. **Guard Active:** The circuit breaker and context compactors are active in the background, consuming zero overhead unless triggered.
|
|
62
|
+
|
|
63
|
+
### What Gets Installed (and How to Remove It)
|
|
64
|
+
|
|
65
|
+
The installer only ever writes to your workspace (or, with `--global`, your home directory) — no daemons, no registry entries, no network services. Depending on which platforms you select, it creates:
|
|
66
|
+
|
|
67
|
+
| File / Directory | Purpose |
|
|
68
|
+
|---|---|
|
|
69
|
+
| `.claude/settings.json` (merged) + `.claude/skills/` + `.claude/agents/` | Claude Code lifecycle hooks, project skills, and named Fable agents |
|
|
70
|
+
| `.cursorrules` | Cursor advisory rules |
|
|
71
|
+
| `.github/copilot-instructions.md` | Copilot Chat advisory instructions |
|
|
72
|
+
| `AGENTS.md` | Codex advisory instructions |
|
|
73
|
+
| `.continue/rules/harness.md` | Continue.dev advisory rules |
|
|
74
|
+
| `.hermes.md` | Hermes Agent advisory context |
|
|
75
|
+
| `.claude/harness-everything/` (or the per-platform equivalent) | Harness state: `manifest.json`, WAL, circuit-breaker counters |
|
|
76
|
+
|
|
77
|
+
The installer records the state directories in `.git/info/exclude` — a local-only git ignore file — so Harness state never lands in a commit and your working tree (including `.gitignore`) is never modified. Everything is removed with the built-in uninstaller:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
npx github:dyphn1/Harness-everything uninstall # interactive
|
|
81
|
+
npx github:dyphn1/Harness-everything uninstall --local --skills -y # non-interactive, workspace only
|
|
82
|
+
npx github:dyphn1/Harness-everything uninstall --global # also remove ~/.agents etc.
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Visualizing the Flow
|
|
88
|
+
|
|
89
|
+
### Without Harness (Endless Trial-and-Error Loop)
|
|
90
|
+
```mermaid
|
|
91
|
+
flowchart TD
|
|
92
|
+
U([User Request]) --> A[AI Coding Agent]
|
|
93
|
+
A -->|Command/Edit| Env[Workspace Environment]
|
|
94
|
+
Env -->|Error / Failure| A
|
|
95
|
+
A -->|Tweak & Retry 1| Env
|
|
96
|
+
Env -->|Error / Failure| A
|
|
97
|
+
A -->|Tweak & Retry 2| Env
|
|
98
|
+
Env -->|Error / Failure| A
|
|
99
|
+
A -->|Tweak & Retry 3... N| Env
|
|
100
|
+
style A fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### With Harness (Guarded and Routed Execution)
|
|
104
|
+
```mermaid
|
|
105
|
+
flowchart TD
|
|
106
|
+
U([User Request]) --> H_Router[Harness Router]
|
|
107
|
+
H_Router -->|Tier 1: Trivial| T1[Direct Edit]
|
|
108
|
+
H_Router -->|Tier 2: Standard| T2["Native TODO / Markdown checklist"]
|
|
109
|
+
H_Router -->|Tier 3: Macro| T3[Fable Multi-Agent Flow]
|
|
110
|
+
|
|
111
|
+
T2 & T3 --> Exec[Execute Code / Run Commands]
|
|
112
|
+
Exec --> Gate{verify-gate.js}
|
|
113
|
+
|
|
114
|
+
Gate -->|Exit 1: Fix & retry| Exec
|
|
115
|
+
Gate -->|Exit 1 - Repeated| CB{Circuit Breaker rule-of-3.js}
|
|
116
|
+
CB -->|Fails 3x| ZO[Zoom Out: Ask Human / Reflect]
|
|
117
|
+
ZO -->|Fresh Diagnosis| Exec
|
|
118
|
+
Gate -->|Exit 0: Success| Done[Checklist item complete]
|
|
119
|
+
|
|
120
|
+
Done --> SE[Self-Evolve: Update Rules]
|
|
121
|
+
style H_Router fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000000
|
|
122
|
+
style CB fill:#fff9c4,stroke:#fbc02d,stroke-width:1px,color:#000000
|
|
123
|
+
style ZO fill:#ffcc80,stroke:#ef6c00,stroke-width:1px,color:#000000
|
|
124
|
+
style Gate fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## Core Modules & Concepts
|
|
130
|
+
|
|
131
|
+
Harness operates through six core cognitive concepts:
|
|
132
|
+
|
|
133
|
+
1. **Router (`tier-router.js`):** Prevents over-engineering. Triages incoming tasks into Tiers: Tier 1 (Direct Edit, no plans), Tier 2 (Standard TDD enforcement), or Tier 3 (Macro Multi-Agent planning and delegation). It also scans every installed platform's `manifest.json` for skills `self-evolve` has generated dynamically, and auto-surfaces the ones whose keyword triggers match the current prompt — so a lesson learned in one session gets suggested again without the human having to remember it exists. If nothing matches at all — including nothing already kept from the open skills ecosystem — `find-skills` checks `npx skills list` live and, if still nothing, searches `skills.sh`/`npx skills`. With explicit approval it defaults to applying the result ephemerally (a content-addressed OS-temp cache, zero footprint after the OS reclaims it) and only permanently installs via `npx skills add` when the user says they'll reuse it — neither path is cached into `manifest.json` the way `generated[]` is, since third-party content isn't lifecycle-owned by Harness.
|
|
134
|
+
2. **Guard (`rule-of-3.js`):** The fail-safe circuit breaker. Tracks failure signatures across terminal runs. If a test or command fails 3 times with the same signature, it locks mutating tools and forces a `zoom-out` reflection: re-verify every assumption with read-only tools, write a fact-checked report, then resume on a fresh diagnosis. The human partner is pulled in only for genuine decisions — or when the same signature trips the breaker a second time. A companion `Stop` gate (`stop-gate.js`) bounces the end of a turn once per edit batch when edits were never followed by a successful verification command.
|
|
135
|
+
3. **Memory (`state-persist.js`):** Session transaction logging. Stores a local Write-Ahead Log (WAL) of milestones, preventing agents from forgetting their current task state if a session limits out or restarts.
|
|
136
|
+
4. **Reflection (`self-evolve`):** Long-term workspace immunization. Upon task completion, the agent reflects on the root cause of resolved issues, then judges whether the lesson is a simple rule or a reusable, complex pattern: simple rules are appended to local workspace rules (`RULES.md`); genuinely reusable patterns are instead packaged as a dynamic skill (via `skill-creator`'s Dynamic Skill Generation Contract) and registered in `manifest.json` so the Router picks it up in future sessions. Either path is validated by a hermetic self-regression suite before it's persisted.
|
|
137
|
+
5. **Subagent Scope Guard (`subagent-scope-guard.js`):** Diffs the whole repo's `git status` before and after every subagent (`Task`) burst, not just the files it was briefed to touch. Catches a subagent that was told to only read/verify but edited files anyway — a real failure mode, not a hypothetical one.
|
|
138
|
+
6. **Cognitive Laws (Agent Cognitive OS):** Six governing laws — Intent Precedence, State Handoff Awakening, Elimination & Prediction, Evidence Assertion, Adversarial Falsification, and Code-Documentation Alignment — are woven directly into the specific skill phase each one governs (e.g. Evidence Assertion inside `tdd`'s RED/GREEN/REFACTOR gates, State Handoff Awakening inside `zoom-out`'s reflection phase) rather than duplicated in one shared file. See the relevant skill's own `SKILL.md` for where a given law actually applies.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Supported AI IDEs & Tools
|
|
143
|
+
|
|
144
|
+
**Only Claude Code gets the hard-boundary hooks.** Every other platform below has no hook/exit-code execution mechanism, so `harness-everything` can only inject advisory text — same protection level as the "Prompt-Only" column in the comparison table above. There is no circuit breaker, no preflight audit, and no WAL on those platforms unless Claude Code (or another hook-capable tool) is also driving the same repo.
|
|
145
|
+
|
|
146
|
+
| AI Agent Tool | Integration Method | Local Target Location | Enforcement |
|
|
147
|
+
|---|---|---|---|
|
|
148
|
+
| **Claude Code** | Native Lifecycle Hooks (`PreToolUse`, `PostToolUse`, `SessionStart`) | `.claude/settings.json` (project) / `~/.claude/settings.json` (user)<br>*.claude/skills/* (Project Skills) / *~/.claude/skills/* (Global Skills) | **Hard** — hooks can block a tool call (`exit(2)`) |
|
|
149
|
+
| **Cursor** | Native Project Rules | `.cursorrules` | Advisory only |
|
|
150
|
+
| **Copilot Chat** | Custom Instructions | `.github/copilot-instructions.md` | Advisory only |
|
|
151
|
+
| **Codex** | Custom Instructions (`AGENTS.md`, not `.codex/config.toml` — that file controls CLI/sandbox behavior, not prompt content) | `AGENTS.md` | Advisory only |
|
|
152
|
+
| **Continue.dev** | Native project rules (a dedicated Markdown file with YAML frontmatter, `alwaysApply: true`) | `.continue/rules/harness.md` (project) / `~/.continue/rules/harness.md` (user, via `--global`) | Advisory only |
|
|
153
|
+
| **Hermes Agent** ([Nous Research](https://hermes-agent.nousresearch.com/)) | Auto-loaded project context file (Hermes also reads `AGENTS.md`/`CLAUDE.md`/`.cursorrules` from the same directory if present, truncated at ~20k chars) | `.hermes.md` (project only — Hermes has no documented global project-instructions equivalent, so `--global --hermes` is a documented no-op) | Advisory only |
|
|
154
|
+
|
|
155
|
+
---
|
|
156
|
+
|
|
157
|
+
## Repository Index
|
|
158
|
+
|
|
159
|
+
## Multi-Agent Workspace
|
|
160
|
+
|
|
161
|
+
Use the canonical `multi-agent-workspace` skill for permanent multi-agent
|
|
162
|
+
infrastructure:
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
node multi-agent-workspace/scripts/scaffold.js --workspace . \
|
|
166
|
+
--agency-source <path-to-agency-agents> --division engineering --platform codex
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
The source is read-only input. The generated `.harness/multi-agent/` directory
|
|
170
|
+
records the six zones, selected metadata, supported converter targets,
|
|
171
|
+
launcher, immutable router, memory index, and structured handoff. Omit the
|
|
172
|
+
source for an explicit unavailable-catalog fallback; do not treat it as a
|
|
173
|
+
complete roster.
|
|
174
|
+
|
|
175
|
+
This repo uses a flat layout (waza/agentskills.io convention). The table below maps each top-level directory to its role.
|
|
176
|
+
|
|
177
|
+
| Directory | Category | Description |
|
|
178
|
+
|---|---|---|
|
|
179
|
+
| `harness-everything` | **Core Runtime** | Bootstrap, tier-router, verify-gate, self-heal |
|
|
180
|
+
| `hooks` | **Core Runtime** | Claude Code lifecycle hooks (circuit breaker, scope guard, stop gate, etc.) |
|
|
181
|
+
| `scripts` | **Core Runtime** | Installer, manifest, prompts, workspace utilities |
|
|
182
|
+
| `bin` | **Core Runtime** | `harness` CLI entry point |
|
|
183
|
+
| `ci` | **Quality Gates** | Consistency check, description collision, mechanism tests, negative controls |
|
|
184
|
+
| `.github` | **CI/CD** | GitHub Actions workflows (ci.yml, release.yml, behavioral-evals.yml) |
|
|
185
|
+
| `.claude-plugin` | **Distribution** | Plugin manifests for Claude Code marketplace |
|
|
186
|
+
| `evals` | **Routing Evals** | 26 trigger/routing eval suites (waza format) |
|
|
187
|
+
| `behavioral-evals` | **Behavioral Evals** | LLM-level discipline cases (headless agent sessions) |
|
|
188
|
+
| `benchmarks` | **Benchmarks** | BENCHMARK_SOP fixtures and recorded A/B results |
|
|
189
|
+
| `docs` | **Documentation** | Philosophy, architecture, routing, reflection, audit |
|
|
190
|
+
| `references` | **Documentation** | Shared checklists (security, performance, definition-of-done) |
|
|
191
|
+
| `multi-agent-workspace` | **Skill (Tier 3)** | Scaffold six zones, select agency specialists, and generate bounded launchers |
|
|
192
|
+
| `environment-detection` | **Foundation** | Preflight: detect OS, shell, package manager |
|
|
193
|
+
| `eval-harness` | **Skill (Tier 2)** | Evaluate agent outputs against rubrics |
|
|
194
|
+
| `fable-discipline` | **Skill (Tier 3)** | Enforce Fable execution guardrails |
|
|
195
|
+
| `fable-mode` | **Skill (Tier 3)** | Multi-agent orchestration with milestone gates |
|
|
196
|
+
| `find-skills` | **Meta** | Discover and install skills from open ecosystems |
|
|
197
|
+
| `git-commit` | **Skill (Tier 1)** | Conventional commit messages with verification |
|
|
198
|
+
| `grill-me` | **Skill (Tier 2)** | Adversarial plan interrogation before implementation |
|
|
199
|
+
| `grill-with-docs` | **Skill (Tier 3)** | Domain-model and decision alignment before design publication |
|
|
200
|
+
| `improve-codebase-architecture` | **Skill (Tier 2)** | Architectural refactoring with evidence |
|
|
201
|
+
| `install-cognitive-os` | **Foundation** | Install Harness cognitive OS into workspace |
|
|
202
|
+
| `repo-docs` | **Skill (Tier 3)** | Generate repository documentation |
|
|
203
|
+
| `rewrite-commits` | **Skill (Tier 1)** | Interactive rebase and commit history cleanup |
|
|
204
|
+
| `security-review` | **Skill (Tier 2)** | OWASP/STRIDE security review |
|
|
205
|
+
| `self-evolve` | **Skill (Tier 2)** | Workspace immunization via dynamic skills |
|
|
206
|
+
| `skill-creator` | **Meta** | Create new skills from patterns |
|
|
207
|
+
| `skill-style` | **Meta** | Skill authoring style guide |
|
|
208
|
+
| `tdd` | **Skill (Tier 2)** | Test-driven development enforcement |
|
|
209
|
+
| `to-spec` | **Advisory (Tier 2/3)** | Publish specs from settled conversations |
|
|
210
|
+
| `to-tickets` | **Advisory (Tier 2/3)** | Decompose settled specs into tracked tickets |
|
|
211
|
+
| `todo-driven-workflow` | **Foundation** | Native TODO or Markdown checklist discipline |
|
|
212
|
+
| `using-git-worktrees` | **Skill (Tier 2)** | Git worktree concurrency patterns |
|
|
213
|
+
| `verification-loop` | **Skill (Tier 2)** | Verify-before-complete enforcement |
|
|
214
|
+
| `verify-before-claim` | **Always-on discipline** | Fact-audit before asserting claims |
|
|
215
|
+
| `zoom-out` | **Circuit breaker** | Circuit-breaker reflection protocol |
|
|
216
|
+
| `opencode-plugin` | **Platform Plugin** | Enforcement hooks for opencode (post-edit, pre-complete, circuit-breaker) |
|
|
217
|
+
|
|
218
|
+
---
|
|
219
|
+
|
|
220
|
+
## Deeper Documentation
|
|
221
|
+
|
|
222
|
+
For a deep dive into individual modules and the underlying philosophy, explore our sub-documents:
|
|
223
|
+
|
|
224
|
+
* [Harness Philosophy](docs/philosophy.md): The core behavior-first, intervention-only design.
|
|
225
|
+
* [Harness Architecture](docs/architecture.md): Lifecycle hooks, security model, and data locality.
|
|
226
|
+
* [Harness Routing & Triage](docs/routing.md): Detailed trigger criteria for Tiers 1, 2, and 3.
|
|
227
|
+
* [Harness Reflection & Memory](docs/reflection.md): WAL session handoffs and workspace rules immunization.
|
|
228
|
+
* [Harness Audit Log](docs/audit.md): Dated self-audit scorecards, methodology, and per-cycle change log.
|
|
229
|
+
|
|
230
|
+
Fable model selection is documented in [fable-mode/references/model-matrix.md](fable-mode/references/model-matrix.md); the explicit entrypoints are `fable-haiku`, `fable-sonnet`, and `fable-opus`.
|
|
231
|
+
|
|
232
|
+
---
|
|
233
|
+
|
|
234
|
+
## Benchmarks & Testing
|
|
235
|
+
|
|
236
|
+
**If you are an agent asked to verify a Harness install, start at [VERIFICATION.md](VERIFICATION.md), not here.** It gives exact commands with exact expected output — install artifact checks for every platform, mechanism-level checks (Claude Code only), the behavioral test prompts below, and an acceptance scorecard to fill in. Do not report "it works" from reading the code — every check there names a command to actually run.
|
|
237
|
+
|
|
238
|
+
`npm test` (`self-evolve/scripts/self-regression.js`) runs deterministic syntax, CLI, routing-matrix, positive skill-route coverage, reference, behavioral-case, Fable model-mode, and mechanism checks (`ci/mechanism-test.js`, `npm run test:mechanism` to run it alone). The suite checks real exit codes and stderr, not just "the code looks right." It runs in CI on every push and pull request (`.github/workflows/ci.yml`); live model evaluations remain explicitly on-demand.
|
|
239
|
+
|
|
240
|
+
For a fuller vanilla-vs-Harness behavioral comparison, see [Harness Skills Benchmark SOP](BENCHMARK_SOP.md) — standardized, reproducible scenarios:
|
|
241
|
+
* **Test A:** Over-engineering defense (Tier 1 typo correction)
|
|
242
|
+
* **Test B:** Micro-error loop defense (Tier 2 bug resolution)
|
|
243
|
+
* **Test C:** Attention loss and hallucination (Tier 3 module refactoring)
|
|
244
|
+
* **Test D:** Knowledge boundary constraints (Offline hallucination prevention)
|
|
245
|
+
* **Test E:** Terminal environment and shell awareness (Windows/Unix shell detection)
|
|
246
|
+
* **Test F** (in VERIFICATION.md, not BENCHMARK_SOP.md): fact-audit discipline — does the agent verify an external-behavior claim before asserting it?
|
|
247
|
+
|
|
248
|
+
Benchmark **results** are tracked in [benchmarks/](benchmarks/README.md) (`run.js scaffold` builds the fixture, `record` commits a schema-validated result bound to a session log). Until those cells are filled, effectiveness claims are unbacked by recorded evidence.
|
|
249
|
+
|
|
250
|
+
### Behavioral evals (LLM-level, on demand)
|
|
251
|
+
|
|
252
|
+
Mechanism tests prove the hooks enforce gates; only live sessions prove agents follow the disciplines. [behavioral-evals/](behavioral-evals/README.md) runs discipline cases (including pressure variants like "we ship in 5 minutes, skip checks") against headless agent sessions (`claude -p`, opencode) in throwaway workspaces: `npm run eval:behavioral`. Token-costing by design — never wired into CI.
|
|
253
|
+
|
|
254
|
+
### Catalog hygiene
|
|
255
|
+
|
|
256
|
+
`npm run test:consistency` keeps the distribution manifests, docs links, skill frontmatter, and routing-eval coverage in lockstep with what is actually on disk; `npm run test:references` checks every executable/deep-dive path named by `SKILL.md`; `npm run test:release` compares the skill catalog and references with `v0.3.3-beta`; and `harness verify-install` detects stale installed versions or file trees. `npm run test:collision` fails CI when two skills' descriptions overlap enough to confuse the router. The repository checks run on every push (`.github/workflows/ci.yml`).
|
|
257
|
+
|
|
258
|
+
---
|
|
259
|
+
|
|
260
|
+
## 📊 System Evaluation
|
|
261
|
+
|
|
262
|
+
Harness audits itself on a dated cycle by running its own test suite and VERIFICATION.md recipes — never by reading the code and assuming it works. The full scorecards, methodology, and per-cycle change log live in [docs/audit.md](docs/audit.md).
|
|
263
|
+
|
|
264
|
+
**Latest local baseline — 2026-09-01**: 26/26 on-disk skills, 26/26 routing-eval directories, and the local npm gates are green. Waza remains a CI-only gate in this checkout because its installer is not available as an npm package; run it on an LF-normalized export as documented by CI.
|
|
265
|
+
|
|
266
|
+
Measure on an LF export, not a Windows working tree — CRLF can inflate waza's token counts and trigger false budget failures.
|
|
267
|
+
|
|
268
|
+
---
|
|
269
|
+
|
|
270
|
+
## 🤝 For Contributors
|
|
271
|
+
|
|
272
|
+
To contribute to Harness or modify any Skill behavior, ensure you run the local self-regression suite first:
|
|
273
|
+
|
|
274
|
+
```bash
|
|
275
|
+
# Run full hermetic static syntax & routing simulations
|
|
276
|
+
npm run self-regression
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
All script modifications must pass 100% cleanly before pushing to keep the runtime immunized against behavioral regression.
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Reporting a Vulnerability
|
|
4
|
+
|
|
5
|
+
If you discover a security vulnerability in Harness, please report it responsibly.
|
|
6
|
+
|
|
7
|
+
**Do not open a public issue for security vulnerabilities.**
|
|
8
|
+
|
|
9
|
+
Instead, email: dyphn1@hotmail.com
|
|
10
|
+
|
|
11
|
+
Include:
|
|
12
|
+
- Description of the vulnerability
|
|
13
|
+
- Steps to reproduce
|
|
14
|
+
- Potential impact
|
|
15
|
+
|
|
16
|
+
## Response Timeline
|
|
17
|
+
|
|
18
|
+
- Acknowledgment within 48 hours
|
|
19
|
+
- Fix or mitigation plan within 7 days for confirmed vulnerabilities
|
|
20
|
+
|
|
21
|
+
## Scope
|
|
22
|
+
|
|
23
|
+
This policy covers the Harness-everything repository and its published npm package.
|
|
24
|
+
|
|
25
|
+
## Hooks and Runtime Behavior
|
|
26
|
+
|
|
27
|
+
Harness installs execution hooks (circuit breaker, scope guard, stop gate) into Claude Code sessions. These hooks:
|
|
28
|
+
|
|
29
|
+
- Enforce file-scope boundaries
|
|
30
|
+
- Track verification state
|
|
31
|
+
- Block unsafe operations
|
|
32
|
+
|
|
33
|
+
If you find a bypass or unintended behavior in these mechanisms, please report it.
|