harness-everything 0.2.2 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +27 -0
- package/.claude-plugin/plugin.json +51 -0
- package/.gitattributes +1 -0
- package/.github/workflows/behavioral-evals.yml +36 -0
- package/.github/workflows/ci.yml +117 -0
- package/.github/workflows/release.yml +45 -0
- package/.husky/commit-msg +1 -0
- package/.waza.yaml +22 -0
- package/AGENTS.md +60 -2
- package/BENCHMARK_SOP.md +62 -62
- package/CHANGELOG.md +297 -114
- package/CODE_OF_CONDUCT.md +24 -0
- package/CONTRIBUTING.md +47 -0
- package/Dockerfile +18 -18
- package/LICENSE +201 -201
- package/README.md +279 -197
- package/SECURITY.md +33 -0
- package/VERIFICATION.md +394 -347
- package/ab-test-deep-analysis-2026-08-27.md +383 -0
- package/ab-test-results-2026-08-27.md +152 -0
- package/ab-test-statistical-analysis-31cases.md +193 -0
- package/behavioral-evals/README.md +85 -0
- package/behavioral-evals/analyze-plugin-results.js +110 -0
- package/behavioral-evals/cases/baseline-add-feature.yaml +34 -0
- package/behavioral-evals/cases/baseline-api-design.yaml +32 -0
- package/behavioral-evals/cases/baseline-debugging.yaml +31 -0
- package/behavioral-evals/cases/baseline-dependency.yaml +24 -0
- package/behavioral-evals/cases/baseline-documentation.yaml +34 -0
- package/behavioral-evals/cases/baseline-error-handling.yaml +31 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +33 -0
- package/behavioral-evals/cases/baseline-performance-monitoring.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance.yaml +34 -0
- package/behavioral-evals/cases/baseline-refactor-with-tests.yaml +35 -0
- package/behavioral-evals/cases/baseline-refactor.yaml +27 -0
- package/behavioral-evals/cases/baseline-security-review.yaml +28 -0
- package/behavioral-evals/cases/baseline-simple-bugfix.yaml +33 -0
- package/behavioral-evals/cases/baseline-tdd-workflow.yaml +28 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +50 -0
- package/behavioral-evals/cases/pressure-budget-skip-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-bypass-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +37 -0
- package/behavioral-evals/cases/pressure-expert-skip-testing.yaml +31 -0
- package/behavioral-evals/cases/pressure-fatigue-skip.yaml +33 -0
- package/behavioral-evals/cases/pressure-management-skip-all.yaml +28 -0
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +28 -0
- package/behavioral-evals/cases/pressure-skip-docs.yaml +50 -0
- package/behavioral-evals/cases/pressure-skip-error-handling.yaml +35 -0
- package/behavioral-evals/cases/pressure-skip-security.yaml +32 -0
- package/behavioral-evals/cases/pressure-skip-tests.yaml +27 -0
- package/behavioral-evals/cases/pressure-skip-verification.yaml +43 -0
- package/behavioral-evals/cases/pressure-social-skip-review.yaml +28 -0
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +47 -0
- package/behavioral-evals/cases/scope-discipline.yaml +32 -0
- package/behavioral-evals/cases/tier1-no-overplan.yaml +26 -0
- package/behavioral-evals/cases/verify-before-done.yaml +31 -0
- package/behavioral-evals/results/.gitkeep +0 -0
- package/behavioral-evals/results/2026-08-23-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-skip-verification.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-sunk-cost-retry.json +20 -0
- package/behavioral-evals/results/2026-08-23-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-23-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-23-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-api-design.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-debugging.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-dependency.json +16 -0
- package/behavioral-evals/results/2026-08-27-baseline-documentation.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-git-commit.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance-monitoring.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor-with-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-security-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-simple-bugfix.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-tdd-workflow.json +24 -0
- package/behavioral-evals/results/2026-08-27-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-budget-skip-review.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-bypass-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-complexity-skip.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-expert-skip-testing.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-fatigue-skip.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-management-skip-all.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-docs.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-security.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-verification.json +24 -0
- package/behavioral-evals/results/2026-08-27-pressure-social-skip-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-sunk-cost-retry.json +24 -0
- package/behavioral-evals/results/2026-08-27-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-27-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-27-verify-before-done.json +20 -0
- package/behavioral-evals/results/run-20260827-115509.log +287 -0
- package/behavioral-evals/run-with-plugin.js +316 -0
- package/behavioral-evals/run.js +587 -0
- package/benchmarks/README.md +63 -0
- package/benchmarks/ab-test-2026-08-26/results.json +55 -0
- package/benchmarks/results/logs/.gitkeep +0 -0
- package/benchmarks/results/logs/test-a-harness.md +31 -0
- package/benchmarks/results/logs/test-a-vanilla.md +28 -0
- package/benchmarks/results/logs/test-b-micro-loop-harness.md +48 -0
- package/benchmarks/results/logs/test-b-micro-loop-vanilla.md +36 -0
- package/benchmarks/results/logs/test-c-attention-loss-harness.md +40 -0
- package/benchmarks/results/logs/test-c-attention-loss-vanilla.md +63 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-harness.md +24 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-vanilla.md +29 -0
- package/benchmarks/results/logs/test-e-shell-awareness-harness.md +20 -0
- package/benchmarks/results/logs/test-e-shell-awareness-vanilla.md +20 -0
- package/benchmarks/results/logs/test-f-fact-audit-harness.md +16 -0
- package/benchmarks/results/logs/test-f-fact-audit-vanilla.md +16 -0
- package/benchmarks/results/results.json +158 -0
- package/benchmarks/run.js +220 -0
- package/benchmarks/schema.json +44 -0
- package/bin/cli.js +152 -97
- package/ci/ab-test-cases/grill-me-adversarial.yaml +18 -0
- package/ci/ab-test-cases/tdd-test-first.yaml +23 -0
- package/ci/ab-test-cases/verification-loop-gates.yaml +25 -0
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +18 -0
- package/ci/ab-test-harness.js +410 -0
- package/{eval-framework → ci}/cases/case1.json +5 -5
- package/ci/cases/case10-small-feature.json +5 -0
- package/ci/cases/case11-readme-typo-update.json +5 -0
- package/ci/cases/case12-chinese-macro-audit.json +5 -0
- package/{eval-framework → ci}/cases/case2.json +5 -5
- package/{eval-framework → ci}/cases/case3.json +5 -5
- package/{eval-framework → ci}/cases/case4-zh-tier2.json +5 -5
- package/{eval-framework → ci}/cases/case5-zh-tier3.json +5 -5
- package/ci/cases/case6-macro-skill-audit.json +5 -0
- package/ci/cases/case7-macro-reference-audit.json +5 -0
- package/ci/cases/case8-controlled-ab-benchmark.json +5 -0
- package/ci/cases/case9-four-issue-audit.json +5 -0
- package/ci/consistency-check.js +347 -0
- package/ci/description-collision.js +109 -0
- package/ci/fable-mode-test.js +120 -0
- package/ci/fixtures/agency-agents/divisions.json +6 -0
- package/ci/fixtures/agency-agents/engineering/code-reviewer.md +11 -0
- package/ci/fixtures/agency-agents/testing/reality-checker.md +11 -0
- package/ci/fixtures/bad-frontmatter-colon/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-a/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-b/SKILL.md +16 -0
- package/ci/mechanism-2a-rule-of-3.test.js +46 -0
- package/ci/mechanism-2b-boundary-guard.test.js +16 -0
- package/ci/mechanism-2c-state-persist.test.js +29 -0
- package/ci/mechanism-2d-fact-audit.test.js +18 -0
- package/ci/mechanism-2e-scope-guard.test.js +24 -0
- package/ci/mechanism-2f-stop-gate.test.js +23 -0
- package/ci/mechanism-2g-platform-ignore.test.js +88 -0
- package/ci/mechanism-2h-installer-manifest.test.js +147 -0
- package/ci/mechanism-2i-crlf-frontmatter.test.js +145 -0
- package/ci/mechanism-2i-install-verification.test.js +34 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +20 -0
- package/ci/mechanism-2k-reference-check.test.js +20 -0
- package/ci/mechanism-2l-multi-agent-workspace.test.js +105 -0
- package/ci/mechanism-test.js +71 -0
- package/ci/negative-controls.js +84 -0
- package/ci/reference-check.js +112 -0
- package/ci/release-consistency-check.js +123 -0
- package/ci/routing-check.js +90 -0
- package/{eval-framework → ci}/runner.js +56 -56
- package/ci/skill-routing-check.js +98 -0
- package/ci/test-helper.js +104 -0
- package/ci/test-negative-controls.js +148 -0
- package/commitlint.config.js +25 -0
- package/docker-verify.sh +266 -266
- package/docs/architecture.md +99 -95
- package/docs/audit.md +53 -0
- package/docs/mechanism-first-skill-mesh.md +163 -0
- package/docs/philosophy.md +47 -45
- package/docs/reflection.md +48 -34
- package/docs/routing.md +58 -45
- package/docs/troubleshooting.md +284 -0
- package/docs/workflows/environment-detection.md +10 -10
- package/docs/workflows/eval-harness.md +13 -6
- package/docs/workflows/fable-discipline.md +9 -9
- package/docs/workflows/fable-mode.md +29 -32
- package/docs/workflows/find-skills.md +36 -0
- package/docs/workflows/git-commit.md +18 -9
- package/docs/workflows/grill-me.md +11 -11
- package/docs/workflows/grill-with-docs.md +14 -7
- package/docs/workflows/harness-everything.md +95 -92
- package/docs/workflows/improve-codebase-architecture.md +15 -7
- package/docs/workflows/install-cognitive-os.md +60 -60
- package/docs/workflows/multi-agent-workspace.md +42 -0
- package/docs/workflows/repo-docs.md +12 -7
- package/docs/workflows/security-review.md +26 -7
- package/docs/workflows/self-evolve.md +27 -44
- package/docs/workflows/skill-creator.md +80 -75
- package/docs/workflows/skill-style.md +64 -64
- package/docs/workflows/tdd.md +42 -39
- package/docs/workflows/to-spec.md +83 -0
- package/docs/workflows/to-tickets.md +80 -0
- package/docs/workflows/todo-driven-workflow.md +34 -36
- package/docs/workflows/using-git-worktrees.md +12 -8
- package/docs/workflows/verification-loop.md +16 -14
- package/docs/workflows/verify-before-claim.md +12 -8
- package/docs/workflows/zoom-out.md +22 -10
- package/environment-detection/SKILL.md +25 -61
- package/environment-detection/references/shell-syntax-rules.md +76 -0
- package/environment-detection/scripts/preflight.js +103 -93
- package/eval-framework/fixtures/bad-frontmatter-colon/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-a/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-b/SKILL.md +14 -0
- package/eval-framework/fixtures/missing-delimiter/SKILL.md +13 -0
- package/eval-framework/fixtures/over-budget-skill/SKILL.md +36 -0
- package/eval-harness/SKILL.md +34 -51
- package/eval-harness/references/scoring-rubric.md +68 -0
- package/eval-harness/scripts/evaluate.js +77 -61
- package/evals/environment-detection/eval.yaml +26 -0
- package/evals/environment-detection/tasks/negative-trigger.yaml +16 -0
- package/evals/environment-detection/tasks/positive-trigger.yaml +15 -0
- package/evals/eval-harness/eval.yaml +26 -0
- package/evals/eval-harness/tasks/negative-trigger.yaml +16 -0
- package/evals/eval-harness/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-discipline/eval.yaml +26 -0
- package/evals/fable-discipline/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-discipline/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-mode/eval.yaml +26 -0
- package/evals/fable-mode/tasks/negative-tier2-no-model.yaml +18 -0
- package/evals/fable-mode/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-mode/tasks/positive-haiku-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-opus-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-sonnet-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-trigger.yaml +15 -0
- package/evals/find-skills/eval.yaml +26 -0
- package/evals/find-skills/tasks/negative-trigger.yaml +16 -0
- package/evals/find-skills/tasks/positive-trigger.yaml +15 -0
- package/evals/git-commit/eval.yaml +26 -0
- package/evals/git-commit/tasks/negative-trigger.yaml +16 -0
- package/evals/git-commit/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-me/eval.yaml +26 -0
- package/evals/grill-me/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-me/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-with-docs/eval.yaml +26 -0
- package/evals/grill-with-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-with-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/harness-everything/eval.yaml +26 -0
- package/evals/harness-everything/tasks/negative-non-software.yaml +16 -0
- package/evals/harness-everything/tasks/negative-skill-indicated.yaml +15 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier2.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier3.yaml +16 -0
- package/evals/improve-codebase-architecture/eval.yaml +26 -0
- package/evals/improve-codebase-architecture/tasks/negative-trigger.yaml +16 -0
- package/evals/improve-codebase-architecture/tasks/positive-trigger.yaml +15 -0
- package/evals/install-cognitive-os/eval.yaml +26 -0
- package/evals/install-cognitive-os/tasks/negative-trigger.yaml +16 -0
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/eval.yaml +26 -0
- package/evals/multi-agent-workspace/tasks/negative-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +15 -0
- package/evals/repo-docs/eval.yaml +26 -0
- package/evals/repo-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/repo-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/rewrite-commits/eval.yaml +26 -0
- package/evals/rewrite-commits/tasks/negative-trigger.yaml +16 -0
- package/evals/rewrite-commits/tasks/positive-trigger.yaml +15 -0
- package/evals/security-review/eval.yaml +26 -0
- package/evals/security-review/tasks/negative-trigger.yaml +16 -0
- package/evals/security-review/tasks/positive-trigger.yaml +15 -0
- package/evals/self-evolve/eval.yaml +26 -0
- package/evals/self-evolve/tasks/negative-trigger.yaml +16 -0
- package/evals/self-evolve/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-creator/eval.yaml +26 -0
- package/evals/skill-creator/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-creator/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-style/eval.yaml +26 -0
- package/evals/skill-style/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-style/tasks/positive-trigger.yaml +15 -0
- package/evals/tdd/eval.yaml +26 -0
- package/evals/tdd/tasks/negative-trivia.yaml +16 -0
- package/evals/tdd/tasks/positive-bugfix.yaml +15 -0
- package/evals/tdd/tasks/positive-feature.yaml +16 -0
- package/evals/tdd/tasks/positive-refactor.yaml +15 -0
- package/evals/to-spec/eval.yaml +26 -0
- package/evals/to-spec/tasks/negative-trigger.yaml +16 -0
- package/evals/to-spec/tasks/positive-trigger.yaml +15 -0
- package/evals/to-tickets/eval.yaml +26 -0
- package/evals/to-tickets/tasks/negative-trigger.yaml +15 -0
- package/evals/to-tickets/tasks/positive-trigger.yaml +15 -0
- package/evals/todo-driven-workflow/eval.yaml +26 -0
- package/evals/todo-driven-workflow/tasks/negative-single-edit.yaml +16 -0
- package/evals/todo-driven-workflow/tasks/positive-multi-step.yaml +15 -0
- package/evals/todo-driven-workflow/tasks/positive-refactor-flow.yaml +15 -0
- package/evals/using-git-worktrees/eval.yaml +26 -0
- package/evals/using-git-worktrees/tasks/negative-trigger.yaml +15 -0
- package/evals/using-git-worktrees/tasks/positive-trigger.yaml +15 -0
- package/evals/verification-loop/eval.yaml +26 -0
- package/evals/verification-loop/tasks/negative-discussion.yaml +16 -0
- package/evals/verification-loop/tasks/positive-pre-ship.yaml +15 -0
- package/evals/verify-before-claim/eval.yaml +26 -0
- package/evals/verify-before-claim/tasks/negative-trigger.yaml +16 -0
- package/evals/verify-before-claim/tasks/positive-trigger.yaml +15 -0
- package/evals/zoom-out/eval.yaml +26 -0
- package/evals/zoom-out/tasks/negative-trivial-edit.yaml +16 -0
- package/evals/zoom-out/tasks/positive-repeated-failure.yaml +15 -0
- package/fable-discipline/SKILL.md +31 -28
- package/fable-discipline/references/discipline-rules.md +22 -0
- package/fable-mode/CONTRACT-FORMAT.md +51 -51
- package/fable-mode/SKILL.md +31 -43
- package/fable-mode/agents/fable-orchestrator.md +99 -92
- package/fable-mode/agents/fable-verifier.md +32 -32
- package/fable-mode/agents/fable-worker-haiku.md +52 -48
- package/fable-mode/agents/fable-worker-sonnet.md +53 -49
- package/fable-mode/execution-guardrails/SKILL.md +43 -87
- package/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/fable-mode/fable-haiku/SKILL.md +36 -48
- package/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/fable-mode/fable-opus/SKILL.md +36 -0
- package/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/fable-mode/model-matrix.json +32 -0
- package/fable-mode/references/execution-phases.md +37 -0
- package/fable-mode/references/model-matrix.md +39 -0
- package/fable-mode/scripts/model-selector.js +129 -0
- package/find-skills/SKILL.md +48 -0
- package/find-skills/references/discovery-flow.md +102 -0
- package/find-skills/scripts/use-skill.js +106 -0
- package/git-commit/SKILL.md +35 -45
- package/git-commit/guides/ANGULAR_STYLE.md +60 -60
- package/git-commit/guides/COMMIT_GENERATION.md +23 -23
- package/git-commit/guides/LANGUAGE_DETECTION.md +16 -16
- package/git-commit/guides/MAIN_REPO.md +14 -14
- package/git-commit/guides/SUBMODULES.md +27 -27
- package/git-commit/references/commit-flow.md +30 -0
- package/grill-me/SKILL.md +34 -27
- package/grill-me/references/grilling-playbook.md +38 -0
- package/grill-with-docs/ADR-FORMAT.md +47 -47
- package/grill-with-docs/CONTEXT-FORMAT.md +63 -63
- package/grill-with-docs/SKILL.md +33 -91
- package/grill-with-docs/references/session-playbook.md +97 -0
- package/harness-everything/SKILL.md +29 -128
- package/harness-everything/examples/README.md +9 -9
- package/harness-everything/examples/todo-workflow.md +22 -22
- package/harness-everything/references/skill-registry.md +39 -0
- package/harness-everything/references/triage-and-tiers.md +116 -0
- package/harness-everything/scripts/bootstrap.js +203 -140
- package/harness-everything/scripts/routing-keywords.json +320 -0
- package/harness-everything/scripts/self-heal.js +241 -189
- package/harness-everything/scripts/tier-router.js +334 -219
- package/harness-everything/scripts/verify-gate.js +100 -31
- package/hooks/hooks.json +155 -155
- package/hooks/scripts/atomic-commit-check.js +68 -68
- package/hooks/scripts/boundary-guard.js +55 -55
- package/hooks/scripts/context-compact.js +114 -114
- package/hooks/scripts/contract-test.js +79 -79
- package/hooks/scripts/depth-guard.js +65 -65
- package/hooks/scripts/lib/harness-state.js +139 -228
- package/hooks/scripts/lib/platforms/claude.js +72 -19
- package/hooks/scripts/lib/platforms/codex.js +24 -6
- package/hooks/scripts/lib/platforms/continue.js +23 -5
- package/hooks/scripts/lib/platforms/copilot.js +23 -5
- package/hooks/scripts/lib/platforms/cursor.js +23 -5
- package/hooks/scripts/lib/platforms/hermes.js +57 -57
- package/hooks/scripts/lib/platforms/index.js +17 -17
- package/hooks/scripts/lib/platforms/worktrees.js +20 -20
- package/hooks/scripts/reset-circuit-breaker.js +30 -30
- package/hooks/scripts/rule-of-3-tracker.js +124 -91
- package/hooks/scripts/rule-of-3.js +187 -176
- package/hooks/scripts/state-persist.js +99 -99
- package/hooks/scripts/stop-gate.js +80 -80
- package/hooks/scripts/subagent-scope-guard.js +88 -88
- package/improve-codebase-architecture/SKILL.md +31 -35
- package/improve-codebase-architecture/guides/DEEPENING.md +37 -37
- package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -123
- package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -44
- package/improve-codebase-architecture/guides/LANGUAGE.md +53 -53
- package/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/install-cognitive-os/SKILL.md +23 -80
- package/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/multi-agent-workspace/SKILL.md +37 -0
- package/{create-agent-launcher → multi-agent-workspace}/examples/csharp-monorepo.md +123 -123
- package/{create-agent-launcher → multi-agent-workspace}/examples/typescript-cli.md +126 -126
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-claude.md +77 -77
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-codex.md +146 -146
- package/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-copilot.md +74 -74
- package/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/role-capabilities.md +108 -108
- package/multi-agent-workspace/references/agency-agents.md +31 -0
- package/multi-agent-workspace/references/architecture-guide.md +32 -0
- package/multi-agent-workspace/references/orchestration.md +48 -0
- package/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/multi-agent-workspace/scripts/index_memory.js +51 -0
- package/multi-agent-workspace/scripts/scaffold.js +253 -0
- package/{create-agent-launcher → multi-agent-workspace}/templates/README.md +17 -17
- package/{create-agent-launcher → multi-agent-workspace}/templates/agent-launcher.md +67 -67
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-csharp.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-polyglot.md +61 -61
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-ts.md +56 -56
- package/{create-agent-launcher → multi-agent-workspace}/templates/memory-keeper.md +36 -36
- package/{create-agent-launcher → multi-agent-workspace}/templates/requirement-analyzer.md +65 -65
- package/{create-agent-launcher → multi-agent-workspace}/templates/specialist-agent.md +34 -34
- package/{create-agent-launcher → multi-agent-workspace}/templates/task-verifier.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/tool-maker.md +32 -32
- package/multi-agent-workspace/workflows/01-init.md +22 -0
- package/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/multi-agent-workspace/workflows/03-generation.md +27 -0
- package/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/opencode-plugin/README.md +94 -0
- package/opencode-plugin/hooks/circuit-breaker.js +121 -0
- package/opencode-plugin/hooks/compliance.js +128 -0
- package/opencode-plugin/hooks/post-edit.js +62 -0
- package/opencode-plugin/hooks/pre-complete.js +65 -0
- package/opencode-plugin/hooks/verify.js +140 -0
- package/opencode-plugin/plugin.json +17 -0
- package/opencode-plugin/test-hook-firing.js +198 -0
- package/package.json +50 -22
- package/references/definition-of-done.md +67 -0
- package/references/performance-checklist.md +153 -0
- package/references/platform-advisory-template.md +58 -0
- package/references/security-checklist.md +205 -0
- package/repo-docs/SKILL.md +38 -37
- package/repo-docs/references/process-guide.md +60 -0
- package/repo-docs/templates/agents-template.md +264 -264
- package/repo-docs/templates/knowledge-agents-template.md +50 -50
- package/repo-docs/templates/knowledge-readme-template.md +83 -83
- package/repo-docs/templates/manual-recon.md +117 -117
- package/repo-docs/templates/multi-skills-readme-template.md +72 -72
- package/repo-docs/templates/product-agents-template.md +78 -78
- package/repo-docs/templates/product-readme-template.md +95 -95
- package/repo-docs/templates/readme-template.md +136 -136
- package/repo-docs/templates/single-skill-readme-template.md +70 -70
- package/repo-docs/templates/skills-agents-template.md +60 -60
- package/rewrite-commits/SKILL.md +22 -17
- package/rewrite-commits/references/workflow.md +18 -0
- package/scripts/installer.js +546 -506
- package/scripts/lib/advisory-text.js +8 -4
- package/scripts/lib/claude-hooks.js +18 -5
- package/scripts/lib/gitignore.js +199 -80
- package/scripts/lib/manifest.js +68 -4
- package/scripts/lib/prompts.js +1 -1
- package/scripts/lib/skills.js +87 -1
- package/scripts/verify-install.js +147 -0
- package/security-review/SKILL.md +35 -496
- package/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/security-review/references/security-checklist.md +478 -0
- package/security-review/scripts/audit-secrets.js +84 -0
- package/self-evolve/SKILL.md +38 -45
- package/self-evolve/references/memory-resolution.md +58 -0
- package/self-evolve/scripts/persist-memory.js +139 -40
- package/self-evolve/scripts/register-dynamic-skill.js +251 -0
- package/self-evolve/scripts/self-regression.js +110 -113
- package/skill-creator/SKILL.md +34 -76
- package/skill-creator/references/authoring-workflow.md +14 -0
- package/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/skill-creator/references/quality-checklist.md +15 -0
- package/skill-creator/references/quality-principles.md +81 -81
- package/skill-creator/references/testing-workflow.md +27 -32
- package/skill-style/SKILL.md +33 -39
- package/skill-style/references/style-guide.md +36 -0
- package/tdd/SKILL.md +34 -43
- package/tdd/guides/deep-modules.md +33 -33
- package/tdd/guides/interface-design.md +31 -31
- package/tdd/guides/mocking.md +59 -59
- package/tdd/guides/refactoring.md +10 -10
- package/tdd/guides/tests.md +61 -61
- package/tdd/references/core-discipline.md +49 -0
- package/to-spec/SKILL.md +42 -0
- package/to-spec/references/process.md +77 -0
- package/to-spec/scripts/check-project-docs.js +259 -0
- package/to-spec/templates/cli-reference.md +31 -0
- package/to-spec/templates/dev-doc.md +27 -0
- package/to-spec/templates/feature-spec.md +59 -0
- package/to-spec/templates/schema-doc.md +31 -0
- package/to-tickets/SKILL.md +42 -0
- package/to-tickets/references/publishing-and-templates.md +97 -0
- package/todo-driven-workflow/SKILL.md +27 -41
- package/todo-driven-workflow/references/execution-guide.md +27 -0
- package/using-git-worktrees/SKILL.md +29 -141
- package/using-git-worktrees/references/workflow-details.md +108 -0
- package/verification-loop/SKILL.md +29 -121
- package/verification-loop/references/verification-phases.md +116 -0
- package/verification-loop/templates/verification-report.template.md +15 -0
- package/verify-before-claim/SKILL.md +27 -70
- package/verify-before-claim/references/verification-guide.md +73 -0
- package/zoom-out/SKILL.md +28 -76
- package/zoom-out/references/circuit-breaker.md +66 -0
- package/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/build-multi-agent-system/AGENTS.md +0 -50
- package/build-multi-agent-system/README.md +0 -63
- package/build-multi-agent-system/SKILL.md +0 -68
- package/create-agent-launcher/SKILL.md +0 -48
- package/create-agent-launcher/guidelines/platform-gemini.md +0 -101
- package/create-agent-launcher/workflows/01-init.md +0 -22
- package/create-agent-launcher/workflows/02-analysis.md +0 -26
- package/create-agent-launcher/workflows/03-generation.md +0 -29
- package/create-agent-launcher/workflows/04-launcher.md +0 -20
- package/docs/workflows/build-multi-agent-system.md +0 -62
- package/docs/workflows/create-agent-launcher.md +0 -62
- package/eval-framework/behavioral-test.js +0 -105
- package/eval-framework/mechanism-test.js +0 -251
- package/harness-everything/scripts/todo-cli.js +0 -157
- package/skill-style/STYLE.md +0 -37
package/docs/reflection.md
CHANGED
|
@@ -1,34 +1,48 @@
|
|
|
1
|
-
# Harness Reflection, Memory & Self-Evolution
|
|
2
|
-
|
|
3
|
-
AI models are notorious for forgetting lessons learned in previous coding sessions. When a bug is solved, if the developer opens a new terminal or launches a new AI session, the agent may encounter the same environment quirks or framework gotchas and make the exact same mistakes again.
|
|
4
|
-
|
|
5
|
-
Harness solves this through its **Self-Evolution and Persistent Memory system**.
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## The Transaction Log (WAL) & Session Handoff
|
|
10
|
-
|
|
11
|
-
Harness maintains a lightweight transaction log (analogous to a Write-Ahead Log in databases) via `hooks/scripts/state-persist.js`.
|
|
12
|
-
|
|
13
|
-
* **Continuous Capture:** Every successful tool use or milestone is recorded in the session transaction log.
|
|
14
|
-
* **Handoff File:** A session handoff checkpoint is saved in the workspace.
|
|
15
|
-
* **Session Start Recovery:** Upon starting a new session, the `bootstrap.js` hook reads the handoff file. It automatically "wakens" the agent, restoring the previous task state, completed milestones, and pending actions. This completely eliminates "session restart amnesia."
|
|
16
|
-
|
|
17
|
-
---
|
|
18
|
-
|
|
19
|
-
## Long-term Memory & Workspace Rules
|
|
20
|
-
|
|
21
|
-
When a complex issue—such as an environment-specific bug, a unique framework quirk, or a custom build command requirement—is successfully resolved:
|
|
22
|
-
|
|
23
|
-
1. **Deep Reflection:** Harness triggers the `self-evolve` skill, directing the model to reflect on the root cause and extract the exact pattern.
|
|
24
|
-
2. **Rule Generation:** The model abstracts the learning into a concise rule (avoiding generic prompt prose).
|
|
25
|
-
3. **
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
1
|
+
# Harness Reflection, Memory & Self-Evolution
|
|
2
|
+
|
|
3
|
+
AI models are notorious for forgetting lessons learned in previous coding sessions. When a bug is solved, if the developer opens a new terminal or launches a new AI session, the agent may encounter the same environment quirks or framework gotchas and make the exact same mistakes again.
|
|
4
|
+
|
|
5
|
+
Harness solves this through its **Self-Evolution and Persistent Memory system**.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## The Transaction Log (WAL) & Session Handoff
|
|
10
|
+
|
|
11
|
+
Harness maintains a lightweight transaction log (analogous to a Write-Ahead Log in databases) via `hooks/scripts/state-persist.js`.
|
|
12
|
+
|
|
13
|
+
* **Continuous Capture:** Every successful tool use or milestone is recorded in the session transaction log.
|
|
14
|
+
* **Handoff File:** A session handoff checkpoint is saved in the workspace.
|
|
15
|
+
* **Session Start Recovery:** Upon starting a new session, the `bootstrap.js` hook reads the handoff file. It automatically "wakens" the agent, restoring the previous task state, completed milestones, and pending actions. This completely eliminates "session restart amnesia."
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Long-term Memory & Workspace Rules
|
|
20
|
+
|
|
21
|
+
When a complex issue—such as an environment-specific bug, a unique framework quirk, or a custom build command requirement—is successfully resolved:
|
|
22
|
+
|
|
23
|
+
1. **Deep Reflection:** Harness triggers the `self-evolve` skill, directing the model to reflect on the root cause and extract the exact pattern.
|
|
24
|
+
2. **Rule Generation:** The model abstracts the learning into a concise rule (avoiding generic prompt prose).
|
|
25
|
+
3. **Memory vs. Dynamic Skill Judgment:** The model decides whether the insight is a simple, localized tip or a reusable, complex procedure — packaging every lesson as a skill would bloat context for no benefit, so this is a deliberate gate, not the default.
|
|
26
|
+
* **Simple Rule (default path):** Appended to local workspace rules (`RULES.md`, or a customized folder) via `persist-memory.js`.
|
|
27
|
+
* **Dynamic Skill (exception path):** Written as a standalone `SKILL.md` under `.claude/harness-everything/skills/generated/<name>/` per `skill-creator`'s Dynamic Skill Generation Contract (with `status: draft` lifecycle metadata and required `triggers:` keywords), then registered via `register-dynamic-skill.js` into `manifest.json`. From there, `tier-router.js` scans and auto-surfaces it in future sessions whenever its triggers match the prompt — closing the loop from "learned once" to "discovered automatically," not just recorded.
|
|
28
|
+
4. **Self-Regression Validation:** Before any new rule or dynamic skill is persisted, Harness executes the `self-regression.js` test suite. This ensures that the generated rules/skills do not conflict with existing core rules and that all script syntax is 100% correct, preventing behavior decay.
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## External Skills: A Deliberately *Different* Loop for Third-Party Content
|
|
33
|
+
|
|
34
|
+
`self-evolve`'s `generated[]` skills are self-authored — this workspace's own agent wrote them, `skill-creator` quality-gates them, and their lifecycle (`draft` → `active` → `deprecated`) is entirely Harness's to track. `find-skills` closes an adjacent gap — a capability that's neither in the static registry nor something this workspace has already learned, but that a third-party skill on `skills.sh` covers — and it was tempting to mirror `generated[]`'s exact mechanism for it (cache the skill's path + triggers in `manifest.json`, let `tier-router.js` auto-surface it the same way).
|
|
35
|
+
|
|
36
|
+
That mirroring was tried and deliberately reverted: `generated[]` is safe to cache because Harness owns the whole lifecycle behind it. A skill fetched via `npx skills add` has none of that guarantee — its author can update the `SKILL.md` at any time, or the user can run `npx skills update`/`remove` independently of Harness, and a cached copy of its `description`/`triggers` from install time would silently drift out of sync with no mechanism to notice. Caching it anyway would launder unaudited, unowned third-party content into the same trust tier as `generated[]`'s self-authored, quality-gated content — a category error, not just a missing feature.
|
|
37
|
+
|
|
38
|
+
So `find-skills` queries live instead of caching bookkeeping, and — a step further — defaults to never installing anything in the first place. `npx skills` already ships an ephemeral mode built for exactly this (`npx skills use <source>`): it fetches a skill and prints its content as a ready-to-follow prompt without installing it anywhere. `find-skills/scripts/use-skill.js` wraps that call behind a content-addressed cache under the OS temp directory (`os.tmpdir()/harness-find-skills-cache/<sha1-of-source>.md`, default 6h freshness) — a real fetch on first lookup, then zero network cost on repeat lookups of the same source, and nothing written to the repo, to a platform's native skill directory, or to `manifest.json`. When the OS eventually reclaims that temp directory, that's the entire "cleanup" story - no explicit expiry logic to maintain, because there was never any permanent state to expire.
|
|
39
|
+
|
|
40
|
+
Permanent install (`npx skills add`) still exists, but only as an explicit exception the user opts into when they know they'll reuse a skill across many future sessions - never the default `find-skills` reaches for. Checking `npx skills list` (§0.3 in `find-skills/SKILL.md`) only ever finds something from that rare path; the default ephemeral path never shows up there, by design, because it never installs. This is the actual answer to "won't skills pile up and bloat context over time?" - most borrowed knowledge is solved-once, applied-once, and leaves nothing behind. Only the deliberate minority a human chooses to keep ever accumulates, and that minority is small enough to stay legible without any Harness-side tracking.
|
|
41
|
+
|
|
42
|
+
---
|
|
43
|
+
|
|
44
|
+
## The Benefits of Self-Evolution
|
|
45
|
+
|
|
46
|
+
* **Immunization:** The workspace is permanently "immunized" against recurring bugs.
|
|
47
|
+
* **Team Alignment:** Because rules are checked into Git, every developer (and every AI agent they launch) immediately benefits from the shared, updated knowledge base.
|
|
48
|
+
* **Token Efficiency:** Rather than the user repeatedly reminding the AI about custom project structures or conventions, the agent reads them natively from the immunized RULES file.
|
package/docs/routing.md
CHANGED
|
@@ -1,45 +1,58 @@
|
|
|
1
|
-
# Harness Routing & Task Triage
|
|
2
|
-
|
|
3
|
-
To prevent both over-engineering and under-planning, Harness classifies every user request into one of three execution tiers. This ensures that trivial tasks are executed instantly without unnecessary bloat, while complex architectural refactoring is strictly designed and verified.
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
##
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
### Tier
|
|
40
|
-
* **Trigger criteria:**
|
|
41
|
-
* **Behavior:** Harness
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
* **
|
|
1
|
+
# Harness Routing & Task Triage
|
|
2
|
+
|
|
3
|
+
To prevent both over-engineering and under-planning, Harness classifies every user request into one of three execution tiers. This ensures that trivial tasks are executed instantly without unnecessary bloat, while complex architectural refactoring is strictly designed and verified.
|
|
4
|
+
|
|
5
|
+
This is a default, not an order: `tier-router.js`'s own output tells the agent to follow its own read of the task when it clearly disagrees, and an explicit instruction from the Human Partner always wins. The keyword heuristic exists to catch the common case cheaply and deterministically, not to override judgment when it's wrong - the tier-router hook and the `zoom-out` step-back-and-retry loop are the safety net for when a hard call is actually needed, not the routing keywords themselves.
|
|
6
|
+
|
|
7
|
+
## Tuning the keyword heuristic
|
|
8
|
+
|
|
9
|
+
`tier-router.js` reads its keyword/guide tables from the sibling `harness-everything/scripts/routing-keywords.json` instead of hardcoding them. Edit that file directly to retune which prompts route to Tier 2/3 or surface a given knowledge guide - no code changes, no re-install, and it travels with `tier-router.js` automatically on every install/update since it lives in the same skill folder. Each `guideGroups` entry matches via a plain-substring `keywords` array (case-insensitive OR) or a `regex` source string; if the file is missing or hand-edited into invalid JSON, the router fails open (empty tables, Tier 1 default) instead of breaking the hook.
|
|
10
|
+
|
|
11
|
+
## Non-Software Task Bypass
|
|
12
|
+
|
|
13
|
+
If the user's prompt is non-software engineering (general Q&A, text translation, web searching, or conversational chat), Harness OS routing is **bypassed completely**. The model replies directly and naturally without emitting a Routing Checkpoint block.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## The Three-Tier Routing System
|
|
18
|
+
|
|
19
|
+
Upon receiving a software engineering request, the `tier-router.js` script analyzes the scope, affected files, and instruction patterns to route the task:
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
[ User Request ]
|
|
23
|
+
|
|
|
24
|
+
[ Software Task? ]
|
|
25
|
+
/ \
|
|
26
|
+
(No) (Yes)
|
|
27
|
+
/ \
|
|
28
|
+
[ Bypass Router ] [ Triage Analysis ]
|
|
29
|
+
Direct Response / | \
|
|
30
|
+
/ | \
|
|
31
|
+
(Trivial) (Standard) (Macro)
|
|
32
|
+
/ | \
|
|
33
|
+
v v v
|
|
34
|
+
[ Tier 1 ] [ Tier 2 ] [ Tier 3 ]
|
|
35
|
+
Direct Edit Test-Driven (TDD) Multi-Agent Flow
|
|
36
|
+
(No plans) (Red-Green-Refactor) (Fable-Mode)
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
### Tier 1: Trivial Tasks (Direct Execution)
|
|
40
|
+
* **Trigger criteria:** Single-file edits, typos, simple styling fixes, readme corrections, or clean-up tasks.
|
|
41
|
+
* **Behavior:** Harness bypasses plans, code-reviews, and sub-agents. The model makes the edit directly, emitting a brief single-line routing checkpoint or direct response.
|
|
42
|
+
* **Verification:** Basic syntax checks or a quick compile verify success.
|
|
43
|
+
|
|
44
|
+
### Tier 2: Standard Tasks (Test-Driven Development)
|
|
45
|
+
* **Trigger criteria:** Normal feature requests, bug fixes, algorithm additions, API endpoint creations, SRP object splitting, or security reviews affecting multiple files.
|
|
46
|
+
* **Behavior:** Harness enforces the `todo-driven-workflow` base execution loop (prioritizing native IDE tools like `manage_todo_list`, with `tasks/todo.md` as the portable fallback) and the `tdd` skill. The model is strictly guided through the Red-Green-Refactor loop:
|
|
47
|
+
1. **Red:** Write an automated test first that reproduces the issue or covers the feature, then run it to verify failure.
|
|
48
|
+
2. **Green:** Implement the minimal amount of code to make the test pass.
|
|
49
|
+
3. **Refactor:** Polish the code, clean up duplicates, and ensure coverage targets are met.
|
|
50
|
+
* **Verification:** Strict execution of the test suite and pre-delivery checks (`verification-loop`).
|
|
51
|
+
|
|
52
|
+
### Tier 3: Macro Tasks (Multi-Agent/Fable-Mode)
|
|
53
|
+
* **Trigger criteria:** High-level design changes, migration of modules, large-scale refactoring, or major capability additions affecting multiple domains.
|
|
54
|
+
* **Behavior:** Harness routes the macro task to `fable-mode` and `multi-agent-workspace`, with the macro plan materialized via `todo-driven-workflow`. Hosts that support sub-agents may then delegate bounded work streams.
|
|
55
|
+
1. **Planning & Alignment:** Perform grilling (`grill-me` / `grill-with-docs`), present an outline preview via `to-spec`, and publish formal specs or vertical-slice tickets (`to-tickets`).
|
|
56
|
+
2. **Delegation:** Specialized sub-agents (or inline persona role-switches) handle specific work streams or audits.
|
|
57
|
+
3. **Handoffs & Memory:** Milestone progress is saved to `todo-driven-workflow` state, and lessons learned are recorded via `self-evolve` (enforcing the 60-line cleanliness and lazy-loading rule).
|
|
58
|
+
* **Verification:** Multi-layer unit, integration, and E2E verification loops (`verification-loop`).
|
|
@@ -0,0 +1,284 @@
|
|
|
1
|
+
# Harness OS Troubleshooting Guide
|
|
2
|
+
|
|
3
|
+
This guide helps you resolve common issues with Harness OS installation and usage.
|
|
4
|
+
|
|
5
|
+
## Installation Issues
|
|
6
|
+
|
|
7
|
+
### Permission Denied Errors
|
|
8
|
+
|
|
9
|
+
**Symptom:** `EACCES: permission denied` during installation
|
|
10
|
+
|
|
11
|
+
**Solution:**
|
|
12
|
+
```bash
|
|
13
|
+
# Option 1: Install without sudo (recommended)
|
|
14
|
+
npx github:dyphn1/Harness-everything install
|
|
15
|
+
|
|
16
|
+
# Option 2: If you need global installation
|
|
17
|
+
npx github:dyphn1/Harness-everything install --global
|
|
18
|
+
|
|
19
|
+
# Option 3: Fix npm permissions (if using npm)
|
|
20
|
+
mkdir ~/.npm-global
|
|
21
|
+
npm config set prefix '~/.npm-global'
|
|
22
|
+
export PATH=~/.npm-global/bin:$PATH
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
### Node.js Version Issues
|
|
26
|
+
|
|
27
|
+
**Symptom:** `SyntaxError` or unsupported engine errors
|
|
28
|
+
|
|
29
|
+
**Solution:**
|
|
30
|
+
```bash
|
|
31
|
+
# Check your Node.js version
|
|
32
|
+
node --version
|
|
33
|
+
|
|
34
|
+
# Harness requires Node.js 18 or higher
|
|
35
|
+
# Update using nvm (recommended):
|
|
36
|
+
nvm install 18
|
|
37
|
+
nvm use 18
|
|
38
|
+
|
|
39
|
+
# Or download from https://nodejs.org/
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
### Network Issues
|
|
43
|
+
|
|
44
|
+
**Symptom:** `npm ERR! network` or timeout errors
|
|
45
|
+
|
|
46
|
+
**Solution:**
|
|
47
|
+
```bash
|
|
48
|
+
# Check your internet connection
|
|
49
|
+
npm ping
|
|
50
|
+
|
|
51
|
+
# Try with different registry
|
|
52
|
+
npm config set registry https://registry.npmjs.org/
|
|
53
|
+
|
|
54
|
+
# Clear npm cache
|
|
55
|
+
npm cache clean --force
|
|
56
|
+
|
|
57
|
+
# Use yarn instead
|
|
58
|
+
yarn install
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Hook Issues
|
|
62
|
+
|
|
63
|
+
### Hooks Not Firing (Claude Code)
|
|
64
|
+
|
|
65
|
+
**Symptom:** Harness rules not being enforced
|
|
66
|
+
|
|
67
|
+
**Solution:**
|
|
68
|
+
1. Verify hooks are installed:
|
|
69
|
+
```bash
|
|
70
|
+
cat .claude/settings.json | grep -A 10 "hooks"
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
2. Check hook file permissions:
|
|
74
|
+
```bash
|
|
75
|
+
ls -la hooks/scripts/*.js
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
3. Restart Claude Code session
|
|
79
|
+
|
|
80
|
+
### Hook Execution Errors
|
|
81
|
+
|
|
82
|
+
**Symptom:** `Error: Cannot find module` or similar
|
|
83
|
+
|
|
84
|
+
**Solution:**
|
|
85
|
+
```bash
|
|
86
|
+
# Verify Node.js is in PATH
|
|
87
|
+
which node
|
|
88
|
+
|
|
89
|
+
# Check if hooks directory exists
|
|
90
|
+
ls -la hooks/scripts/
|
|
91
|
+
|
|
92
|
+
# Reinstall Harness
|
|
93
|
+
npx github:dyphn1/Harness-everything uninstall
|
|
94
|
+
npx github:dyphn1/Harness-everything install
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Circuit Breaker Tripping Too Often
|
|
98
|
+
|
|
99
|
+
**Symptom:** `RULE OF 3 CIRCUIT BREAKER TRIGGERED!` frequently
|
|
100
|
+
|
|
101
|
+
**Solution:**
|
|
102
|
+
1. This is normal behavior - Harness is protecting you from infinite loops
|
|
103
|
+
2. Follow the zoom-out instructions in the error message
|
|
104
|
+
3. If it's a false positive, check the error signature:
|
|
105
|
+
```bash
|
|
106
|
+
cat .claude/harness-everything/state/sessions/default/rule-of-3-state.json
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
4. Reset if needed:
|
|
110
|
+
```bash
|
|
111
|
+
npm run harness:reset
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## Platform-Specific Issues
|
|
115
|
+
|
|
116
|
+
### Cursor Not Showing Rules
|
|
117
|
+
|
|
118
|
+
**Symptom:** `.cursorrules` file not created
|
|
119
|
+
|
|
120
|
+
**Solution:**
|
|
121
|
+
1. Verify file exists:
|
|
122
|
+
```bash
|
|
123
|
+
cat .cursorrules | head -20
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
2. If missing, reinstall with Cursor flag:
|
|
127
|
+
```bash
|
|
128
|
+
npx github:dyphn1/Harness-everything install --cursor
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
3. Restart Cursor
|
|
132
|
+
|
|
133
|
+
### Copilot Not Following Rules
|
|
134
|
+
|
|
135
|
+
**Symptom:** GitHub Copilot ignoring Harness guidance
|
|
136
|
+
|
|
137
|
+
**Solution:**
|
|
138
|
+
1. Check file exists:
|
|
139
|
+
```bash
|
|
140
|
+
cat .github/copilot-instructions.md | head -20
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
2. Ensure Copilot is using the instructions file
|
|
144
|
+
3. Restart VS Code/GitHub Copilot
|
|
145
|
+
|
|
146
|
+
### Codex/AGENTS.md Issues
|
|
147
|
+
|
|
148
|
+
**Symptom:** Codex not recognizing Harness
|
|
149
|
+
|
|
150
|
+
**Solution:**
|
|
151
|
+
1. Check AGENTS.md exists:
|
|
152
|
+
```bash
|
|
153
|
+
cat AGENTS.md | head -20
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
2. Reinstall with Codex flag:
|
|
157
|
+
```bash
|
|
158
|
+
npx github:dyphn1/Harness-everything install --codex
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
## Performance Issues
|
|
162
|
+
|
|
163
|
+
### Slow Hook Execution
|
|
164
|
+
|
|
165
|
+
**Symptom:** Terminal commands feel slower
|
|
166
|
+
|
|
167
|
+
**Solution:**
|
|
168
|
+
1. Check hook execution time:
|
|
169
|
+
```bash
|
|
170
|
+
time node hooks/scripts/rule-of-3.js < /dev/null
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
2. If >200ms, check for:
|
|
174
|
+
- Large state files
|
|
175
|
+
- Network calls in hooks
|
|
176
|
+
- Complex file operations
|
|
177
|
+
|
|
178
|
+
3. Reset state:
|
|
179
|
+
```bash
|
|
180
|
+
rm -rf .claude/harness-everything/state/sessions/
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
### High Memory Usage
|
|
184
|
+
|
|
185
|
+
**Symptom:** Node.js processes consuming too much memory
|
|
186
|
+
|
|
187
|
+
**Solution:**
|
|
188
|
+
1. Check for memory leaks:
|
|
189
|
+
```bash
|
|
190
|
+
# Monitor Node.js processes
|
|
191
|
+
ps aux | grep node
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
2. Restart session if needed
|
|
195
|
+
3. Report issue if persistent
|
|
196
|
+
|
|
197
|
+
## Debugging
|
|
198
|
+
|
|
199
|
+
### Enable Verbose Logging
|
|
200
|
+
|
|
201
|
+
```bash
|
|
202
|
+
# Set debug environment variable
|
|
203
|
+
export HARNESS_DEBUG=1
|
|
204
|
+
|
|
205
|
+
# Run with verbose output
|
|
206
|
+
node hooks/scripts/rule-of-3.js < payload.json
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
### Check State Files
|
|
210
|
+
|
|
211
|
+
```bash
|
|
212
|
+
# View current state
|
|
213
|
+
cat .claude/harness-everything/state/sessions/default/rule-of-3-state.json
|
|
214
|
+
cat .claude/harness-everything/state/sessions/default/handoff-state.json
|
|
215
|
+
|
|
216
|
+
# Reset all state
|
|
217
|
+
rm -rf .claude/harness-everything/state/sessions/
|
|
218
|
+
npm run harness:reset
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
### Manual Hook Testing
|
|
222
|
+
|
|
223
|
+
```bash
|
|
224
|
+
# Test rule-of-3 hook
|
|
225
|
+
echo '{"tool_name":"Bash","tool_input":{"command":"test"}}' | node hooks/scripts/rule-of-3.js
|
|
226
|
+
|
|
227
|
+
# Test state-persist hook
|
|
228
|
+
echo '{"tool_name":"Edit","tool_input":{"file_path":"test.js"}}' | node hooks/scripts/state-persist.js
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
## Getting Help
|
|
232
|
+
|
|
233
|
+
### Collect Diagnostic Information
|
|
234
|
+
|
|
235
|
+
```bash
|
|
236
|
+
# Generate diagnostic report
|
|
237
|
+
echo "=== System Info ===" > harness-diagnostic.txt
|
|
238
|
+
node --version >> harness-diagnostic.txt
|
|
239
|
+
npm --version >> harness-diagnostic.txt
|
|
240
|
+
echo "" >> harness-diagnostic.txt
|
|
241
|
+
|
|
242
|
+
echo "=== Harness Version ===" >> harness-diagnostic.txt
|
|
243
|
+
cat package.json | grep version >> harness-diagnostic.txt
|
|
244
|
+
echo "" >> harness-diagnostic.txt
|
|
245
|
+
|
|
246
|
+
echo "=== Installed Platforms ===" >> harness-diagnostic.txt
|
|
247
|
+
ls -la .claude/ .cursorrules .github/copilot-instructions.md AGENTS.md .continue/rules/ .hermes.md 2>/dev/null >> harness-diagnostic.txt
|
|
248
|
+
echo "" >> harness-diagnostic.txt
|
|
249
|
+
|
|
250
|
+
echo "=== Hook Status ===" >> harness-diagnostic.txt
|
|
251
|
+
cat .claude/settings.json | grep -A 20 "hooks" >> harness-diagnostic.txt 2>/dev/null
|
|
252
|
+
echo "" >> harness-diagnostic.txt
|
|
253
|
+
|
|
254
|
+
echo "=== Recent Errors ===" >> harness-diagnostic.txt
|
|
255
|
+
# Add any recent error logs here
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
### Contact Support
|
|
259
|
+
|
|
260
|
+
1. **GitHub Issues:** https://github.com/dyphn1/Harness-everything/issues
|
|
261
|
+
2. **Include:** Diagnostic report, error messages, steps to reproduce
|
|
262
|
+
3. **Platform:** Specify which AI tool you're using (Claude Code, Cursor, etc.)
|
|
263
|
+
|
|
264
|
+
## Common Error Messages
|
|
265
|
+
|
|
266
|
+
### "Cannot find module './lib/harness-state'"
|
|
267
|
+
- **Cause:** Missing dependencies
|
|
268
|
+
- **Solution:** Run `npm install` or reinstall Harness
|
|
269
|
+
|
|
270
|
+
### "Routing-keywords.json missing/invalid"
|
|
271
|
+
- **Cause:** Configuration file corrupted
|
|
272
|
+
- **Solution:** Will fail open to Tier 1 - reinstall to restore
|
|
273
|
+
|
|
274
|
+
### "Session state not found"
|
|
275
|
+
- **Cause:** First run or state cleared
|
|
276
|
+
- **Solution:** Normal - state will be created on first use
|
|
277
|
+
|
|
278
|
+
### "Zoom-out report not found"
|
|
279
|
+
- **Cause:** Circuit breaker tripped but no report written
|
|
280
|
+
- **Solution:** Follow the zoom-out instructions in the error message
|
|
281
|
+
|
|
282
|
+
---
|
|
283
|
+
|
|
284
|
+
*This troubleshooting guide covers the most common issues. For other problems, please open a GitHub issue with diagnostic information.*
|
|
@@ -10,16 +10,16 @@ This section visualizes how the `environment-detection` skill executes internall
|
|
|
10
10
|
|
|
11
11
|
```mermaid
|
|
12
12
|
graph TD
|
|
13
|
-
Start([
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
13
|
+
Start([Session Start / Discover Phase]) --> CheckNode{Node.js Executable?}
|
|
14
|
+
CheckNode -->|Yes| RunPreflight["Run scripts/preflight.js"]
|
|
15
|
+
CheckNode -->|No / Fails| Heuristic["Inspect <environment_info> & Probe Shell via echo/env"]
|
|
16
|
+
|
|
17
|
+
RunPreflight --> ParseOutput["Parse OS, Shell, Path Separators & CLI Runtimes"]
|
|
18
|
+
Heuristic --> ParseOutput
|
|
19
|
+
|
|
20
|
+
ParseOutput --> SelfHeal["Run self-heal.js if Node available"]
|
|
21
|
+
SelfHeal --> AlignSyntax["Adopt Path Slash (/ vs \\) & Env Var Syntax ($VAR vs %VAR%)"]
|
|
22
|
+
AlignSyntax --> End([Environment Context Established])
|
|
23
23
|
```
|
|
24
24
|
|
|
25
25
|
---
|
|
@@ -10,12 +10,19 @@ This section visualizes how the `eval-harness` skill executes internally, detail
|
|
|
10
10
|
|
|
11
11
|
```mermaid
|
|
12
12
|
graph TD
|
|
13
|
-
Start([
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
13
|
+
Start([Trigger: Benchmark / Evaluate Log]) --> ParseLog["1. Parse Log: Actions, Error Loops, Token Indicator"]
|
|
14
|
+
ParseLog --> CalcRubric["2. Calculate Scores across 4 Dimensions (0-10)"]
|
|
15
|
+
CalcRubric --> CheckScript{3. Is evaluate.js Executable?}
|
|
16
|
+
|
|
17
|
+
CheckScript -->|Yes| RunScript["Run node eval-harness/scripts/evaluate.js"]
|
|
18
|
+
CheckScript -->|No / Fails| FileFallback{"4. Markdown File Fallback"}
|
|
19
|
+
|
|
20
|
+
FileFallback -->|evals/ Exists| WriteEvals["Save Scorecard to evals/scorecard.md"]
|
|
21
|
+
FileFallback -->|No evals/ Folder| WritePlatform["Save Scorecard to .github/harness-everything/evals/scorecard.md"]
|
|
22
|
+
|
|
23
|
+
RunScript --> End([Evaluation Scorecard Logged])
|
|
24
|
+
WriteEvals --> End
|
|
25
|
+
WritePlatform --> End
|
|
19
26
|
```
|
|
20
27
|
|
|
21
28
|
---
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Workflow: Fable Discipline
|
|
2
2
|
|
|
3
|
-
> Shadow guard for fable-mode preventing context bloat, enforcing physical boundaries, and
|
|
3
|
+
> Shadow guard for fable-mode preventing context bloat, enforcing physical boundaries, and preserving compact stage handoffs.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -10,14 +10,14 @@ This section visualizes how the `fable-discipline` skill executes internally, de
|
|
|
10
10
|
|
|
11
11
|
```mermaid
|
|
12
12
|
graph TD
|
|
13
|
-
Start([Fable Task Initiated]) --> AuditContext["Audit
|
|
14
|
-
AuditContext --> CheckTokenLimit{
|
|
13
|
+
Start([Fable Task Initiated]) --> AuditContext["Audit active context against the stage contract"]
|
|
14
|
+
AuditContext --> CheckTokenLimit{Context exceeds the active budget?}
|
|
15
15
|
CheckTokenLimit -->|Yes| TrimContext["Evict raw logs, completed tasks, and non-essential file contents"]
|
|
16
16
|
CheckTokenLimit -->|No| KeepState["Continue monitoring context state"]
|
|
17
17
|
TrimContext --> EnforceLimit["Enforce physical boundary guidelines"]
|
|
18
18
|
KeepState --> EnforceLimit
|
|
19
|
-
EnforceLimit -->
|
|
20
|
-
|
|
19
|
+
EnforceLimit --> RecordHandoff["Record a compact stage handoff"]
|
|
20
|
+
RecordHandoff --> End([Context maintained under safe, high-attention limits])
|
|
21
21
|
```
|
|
22
22
|
|
|
23
23
|
---
|
|
@@ -41,11 +41,11 @@ Here we model concrete real-world scenarios and use cases of the `fable-discipli
|
|
|
41
41
|
|
|
42
42
|
```mermaid
|
|
43
43
|
graph TD
|
|
44
|
-
Start["Fable-mode is editing a large monorepo
|
|
45
|
-
Monitor --> TriggerPruning["Identifies
|
|
44
|
+
Start["Fable-mode is editing a large monorepo with growing context"] --> Monitor["Fable-discipline analyzes active context"]
|
|
45
|
+
Monitor --> TriggerPruning["Identifies raw logs & completed sub-task details"]
|
|
46
46
|
TriggerPruning --> Clean["Evicts completed step details and logs from immediate active context"]
|
|
47
|
-
Clean --> Compress["Summarizes historical state into
|
|
48
|
-
Compress --> Done([Active context
|
|
47
|
+
Clean --> Compress["Summarizes historical state into the stage handoff"]
|
|
48
|
+
Compress --> Done([Active context remains bounded while preserving required facts])
|
|
49
49
|
```
|
|
50
50
|
|
|
51
51
|
---
|
|
@@ -1,32 +1,29 @@
|
|
|
1
|
-
# Workflow: Fable Mode
|
|
2
|
-
|
|
3
|
-
> Macro-task planning and execution engine. Like all Harness workflows, this is not a straight line to success. It requires continuous verification and course correction.
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## 1. Skill Behavior Workflow
|
|
8
|
-
|
|
9
|
-
This section visualizes how `fable-mode` handles macro tasks. Subagent failure or integration crashes are expected realities, not edge cases.
|
|
10
|
-
|
|
11
|
-
```mermaid
|
|
12
|
-
graph TD
|
|
13
|
-
Start([Tier 3 Macro Task Triggered]) -->
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
FinalCheck -->|Exit 1| Diagnose
|
|
31
|
-
FinalCheck -->|Exit 0| End([Macro rewrite successfully deployed])
|
|
32
|
-
```
|
|
1
|
+
# Workflow: Fable Mode
|
|
2
|
+
|
|
3
|
+
> Macro-task planning and execution engine. Like all Harness workflows, this is not a straight line to success. It requires continuous verification and course correction.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Skill Behavior Workflow
|
|
8
|
+
|
|
9
|
+
This section visualizes how `fable-mode` handles macro tasks. Subagent failure or integration crashes are expected realities, not edge cases.
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
graph TD
|
|
13
|
+
Start([Tier 3 Macro Task Triggered]) --> Discovery["1. Discovery & Architectural Plan"]
|
|
14
|
+
Discovery --> InitTodo["2. Delegate Roadmap to todo-driven-workflow"]
|
|
15
|
+
|
|
16
|
+
InitTodo --> Delegate{3. Delegate Sub-tasks}
|
|
17
|
+
Delegate -->|Sub-agent Tool Available| SpawnSub["Spawn Sub-agent via multi-agent-workspace"]
|
|
18
|
+
Delegate -->|No Sub-agent Tool| InlinePersona["Inline Persona Role-Switch Fallback"]
|
|
19
|
+
|
|
20
|
+
SpawnSub --> ScopeCheck["Monitor via hooks/scripts/subagent-scope-guard.js"]
|
|
21
|
+
InlinePersona --> ScopeCheck
|
|
22
|
+
|
|
23
|
+
ScopeCheck --> VerifyStep["4. Integration Verification via harness-everything/scripts/verify-gate.js"]
|
|
24
|
+
VerifyStep --> CheckGate{Integration Verification Passed?}
|
|
25
|
+
|
|
26
|
+
CheckGate -->|No: Integration Error| FixBlocker["Add Blocker Item to todo-driven-workflow & Fix"] --> VerifyStep
|
|
27
|
+
CheckGate -->|Yes: Gate Passes| Transition["5. Transition Scaffolding to TDD Iteration"]
|
|
28
|
+
Transition --> End([Macro Scaffolding Successfully Deployed])
|
|
29
|
+
```
|