harness-everything 0.2.2 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +27 -0
- package/.claude-plugin/plugin.json +51 -0
- package/.gitattributes +1 -0
- package/.github/workflows/behavioral-evals.yml +36 -0
- package/.github/workflows/ci.yml +117 -0
- package/.github/workflows/release.yml +45 -0
- package/.husky/commit-msg +1 -0
- package/.waza.yaml +22 -0
- package/AGENTS.md +60 -2
- package/BENCHMARK_SOP.md +62 -62
- package/CHANGELOG.md +297 -114
- package/CODE_OF_CONDUCT.md +24 -0
- package/CONTRIBUTING.md +47 -0
- package/Dockerfile +18 -18
- package/LICENSE +201 -201
- package/README.md +279 -197
- package/SECURITY.md +33 -0
- package/VERIFICATION.md +394 -347
- package/ab-test-deep-analysis-2026-08-27.md +383 -0
- package/ab-test-results-2026-08-27.md +152 -0
- package/ab-test-statistical-analysis-31cases.md +193 -0
- package/behavioral-evals/README.md +85 -0
- package/behavioral-evals/analyze-plugin-results.js +110 -0
- package/behavioral-evals/cases/baseline-add-feature.yaml +34 -0
- package/behavioral-evals/cases/baseline-api-design.yaml +32 -0
- package/behavioral-evals/cases/baseline-debugging.yaml +31 -0
- package/behavioral-evals/cases/baseline-dependency.yaml +24 -0
- package/behavioral-evals/cases/baseline-documentation.yaml +34 -0
- package/behavioral-evals/cases/baseline-error-handling.yaml +31 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +33 -0
- package/behavioral-evals/cases/baseline-performance-monitoring.yaml +28 -0
- package/behavioral-evals/cases/baseline-performance.yaml +34 -0
- package/behavioral-evals/cases/baseline-refactor-with-tests.yaml +35 -0
- package/behavioral-evals/cases/baseline-refactor.yaml +27 -0
- package/behavioral-evals/cases/baseline-security-review.yaml +28 -0
- package/behavioral-evals/cases/baseline-simple-bugfix.yaml +33 -0
- package/behavioral-evals/cases/baseline-tdd-workflow.yaml +28 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +50 -0
- package/behavioral-evals/cases/pressure-budget-skip-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-bypass-review.yaml +27 -0
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +37 -0
- package/behavioral-evals/cases/pressure-expert-skip-testing.yaml +31 -0
- package/behavioral-evals/cases/pressure-fatigue-skip.yaml +33 -0
- package/behavioral-evals/cases/pressure-management-skip-all.yaml +28 -0
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +28 -0
- package/behavioral-evals/cases/pressure-skip-docs.yaml +50 -0
- package/behavioral-evals/cases/pressure-skip-error-handling.yaml +35 -0
- package/behavioral-evals/cases/pressure-skip-security.yaml +32 -0
- package/behavioral-evals/cases/pressure-skip-tests.yaml +27 -0
- package/behavioral-evals/cases/pressure-skip-verification.yaml +43 -0
- package/behavioral-evals/cases/pressure-social-skip-review.yaml +28 -0
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +47 -0
- package/behavioral-evals/cases/scope-discipline.yaml +32 -0
- package/behavioral-evals/cases/tier1-no-overplan.yaml +26 -0
- package/behavioral-evals/cases/verify-before-done.yaml +31 -0
- package/behavioral-evals/results/.gitkeep +0 -0
- package/behavioral-evals/results/2026-08-23-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-skip-verification.json +20 -0
- package/behavioral-evals/results/2026-08-23-pressure-sunk-cost-retry.json +20 -0
- package/behavioral-evals/results/2026-08-23-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-23-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-23-verify-before-done.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-api-design.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-debugging.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-dependency.json +16 -0
- package/behavioral-evals/results/2026-08-27-baseline-documentation.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-git-commit.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance-monitoring.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-performance.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor-with-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-refactor.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-security-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-simple-bugfix.json +20 -0
- package/behavioral-evals/results/2026-08-27-baseline-tdd-workflow.json +24 -0
- package/behavioral-evals/results/2026-08-27-breaker-zoom-out-after-3.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-budget-skip-review.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-bypass-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-complexity-skip.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-expert-skip-testing.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-fatigue-skip.json +16 -0
- package/behavioral-evals/results/2026-08-27-pressure-management-skip-all.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-docs.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-error-handling.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-security.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-tests.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-skip-verification.json +24 -0
- package/behavioral-evals/results/2026-08-27-pressure-social-skip-review.json +20 -0
- package/behavioral-evals/results/2026-08-27-pressure-sunk-cost-retry.json +24 -0
- package/behavioral-evals/results/2026-08-27-scope-discipline.json +16 -0
- package/behavioral-evals/results/2026-08-27-tier1-no-overplan.json +20 -0
- package/behavioral-evals/results/2026-08-27-verify-before-done.json +20 -0
- package/behavioral-evals/results/run-20260827-115509.log +287 -0
- package/behavioral-evals/run-with-plugin.js +316 -0
- package/behavioral-evals/run.js +587 -0
- package/benchmarks/README.md +63 -0
- package/benchmarks/ab-test-2026-08-26/results.json +55 -0
- package/benchmarks/results/logs/.gitkeep +0 -0
- package/benchmarks/results/logs/test-a-harness.md +31 -0
- package/benchmarks/results/logs/test-a-vanilla.md +28 -0
- package/benchmarks/results/logs/test-b-micro-loop-harness.md +48 -0
- package/benchmarks/results/logs/test-b-micro-loop-vanilla.md +36 -0
- package/benchmarks/results/logs/test-c-attention-loss-harness.md +40 -0
- package/benchmarks/results/logs/test-c-attention-loss-vanilla.md +63 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-harness.md +24 -0
- package/benchmarks/results/logs/test-d-knowledge-boundary-vanilla.md +29 -0
- package/benchmarks/results/logs/test-e-shell-awareness-harness.md +20 -0
- package/benchmarks/results/logs/test-e-shell-awareness-vanilla.md +20 -0
- package/benchmarks/results/logs/test-f-fact-audit-harness.md +16 -0
- package/benchmarks/results/logs/test-f-fact-audit-vanilla.md +16 -0
- package/benchmarks/results/results.json +158 -0
- package/benchmarks/run.js +220 -0
- package/benchmarks/schema.json +44 -0
- package/bin/cli.js +152 -97
- package/ci/ab-test-cases/grill-me-adversarial.yaml +18 -0
- package/ci/ab-test-cases/tdd-test-first.yaml +23 -0
- package/ci/ab-test-cases/verification-loop-gates.yaml +25 -0
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +18 -0
- package/ci/ab-test-harness.js +410 -0
- package/{eval-framework → ci}/cases/case1.json +5 -5
- package/ci/cases/case10-small-feature.json +5 -0
- package/ci/cases/case11-readme-typo-update.json +5 -0
- package/ci/cases/case12-chinese-macro-audit.json +5 -0
- package/{eval-framework → ci}/cases/case2.json +5 -5
- package/{eval-framework → ci}/cases/case3.json +5 -5
- package/{eval-framework → ci}/cases/case4-zh-tier2.json +5 -5
- package/{eval-framework → ci}/cases/case5-zh-tier3.json +5 -5
- package/ci/cases/case6-macro-skill-audit.json +5 -0
- package/ci/cases/case7-macro-reference-audit.json +5 -0
- package/ci/cases/case8-controlled-ab-benchmark.json +5 -0
- package/ci/cases/case9-four-issue-audit.json +5 -0
- package/ci/consistency-check.js +347 -0
- package/ci/description-collision.js +109 -0
- package/ci/fable-mode-test.js +120 -0
- package/ci/fixtures/agency-agents/divisions.json +6 -0
- package/ci/fixtures/agency-agents/engineering/code-reviewer.md +11 -0
- package/ci/fixtures/agency-agents/testing/reality-checker.md +11 -0
- package/ci/fixtures/bad-frontmatter-colon/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-a/SKILL.md +16 -0
- package/ci/fixtures/duplicate-description/skill-b/SKILL.md +16 -0
- package/ci/mechanism-2a-rule-of-3.test.js +46 -0
- package/ci/mechanism-2b-boundary-guard.test.js +16 -0
- package/ci/mechanism-2c-state-persist.test.js +29 -0
- package/ci/mechanism-2d-fact-audit.test.js +18 -0
- package/ci/mechanism-2e-scope-guard.test.js +24 -0
- package/ci/mechanism-2f-stop-gate.test.js +23 -0
- package/ci/mechanism-2g-platform-ignore.test.js +88 -0
- package/ci/mechanism-2h-installer-manifest.test.js +147 -0
- package/ci/mechanism-2i-crlf-frontmatter.test.js +145 -0
- package/ci/mechanism-2i-install-verification.test.js +34 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +20 -0
- package/ci/mechanism-2k-reference-check.test.js +20 -0
- package/ci/mechanism-2l-multi-agent-workspace.test.js +105 -0
- package/ci/mechanism-test.js +71 -0
- package/ci/negative-controls.js +84 -0
- package/ci/reference-check.js +112 -0
- package/ci/release-consistency-check.js +123 -0
- package/ci/routing-check.js +90 -0
- package/{eval-framework → ci}/runner.js +56 -56
- package/ci/skill-routing-check.js +98 -0
- package/ci/test-helper.js +104 -0
- package/ci/test-negative-controls.js +148 -0
- package/commitlint.config.js +25 -0
- package/docker-verify.sh +266 -266
- package/docs/architecture.md +99 -95
- package/docs/audit.md +53 -0
- package/docs/mechanism-first-skill-mesh.md +163 -0
- package/docs/philosophy.md +47 -45
- package/docs/reflection.md +48 -34
- package/docs/routing.md +58 -45
- package/docs/troubleshooting.md +284 -0
- package/docs/workflows/environment-detection.md +10 -10
- package/docs/workflows/eval-harness.md +13 -6
- package/docs/workflows/fable-discipline.md +9 -9
- package/docs/workflows/fable-mode.md +29 -32
- package/docs/workflows/find-skills.md +36 -0
- package/docs/workflows/git-commit.md +18 -9
- package/docs/workflows/grill-me.md +11 -11
- package/docs/workflows/grill-with-docs.md +14 -7
- package/docs/workflows/harness-everything.md +95 -92
- package/docs/workflows/improve-codebase-architecture.md +15 -7
- package/docs/workflows/install-cognitive-os.md +60 -60
- package/docs/workflows/multi-agent-workspace.md +42 -0
- package/docs/workflows/repo-docs.md +12 -7
- package/docs/workflows/security-review.md +26 -7
- package/docs/workflows/self-evolve.md +27 -44
- package/docs/workflows/skill-creator.md +80 -75
- package/docs/workflows/skill-style.md +64 -64
- package/docs/workflows/tdd.md +42 -39
- package/docs/workflows/to-spec.md +83 -0
- package/docs/workflows/to-tickets.md +80 -0
- package/docs/workflows/todo-driven-workflow.md +34 -36
- package/docs/workflows/using-git-worktrees.md +12 -8
- package/docs/workflows/verification-loop.md +16 -14
- package/docs/workflows/verify-before-claim.md +12 -8
- package/docs/workflows/zoom-out.md +22 -10
- package/environment-detection/SKILL.md +25 -61
- package/environment-detection/references/shell-syntax-rules.md +76 -0
- package/environment-detection/scripts/preflight.js +103 -93
- package/eval-framework/fixtures/bad-frontmatter-colon/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-a/SKILL.md +14 -0
- package/eval-framework/fixtures/duplicate-desc-skill-b/SKILL.md +14 -0
- package/eval-framework/fixtures/missing-delimiter/SKILL.md +13 -0
- package/eval-framework/fixtures/over-budget-skill/SKILL.md +36 -0
- package/eval-harness/SKILL.md +34 -51
- package/eval-harness/references/scoring-rubric.md +68 -0
- package/eval-harness/scripts/evaluate.js +77 -61
- package/evals/environment-detection/eval.yaml +26 -0
- package/evals/environment-detection/tasks/negative-trigger.yaml +16 -0
- package/evals/environment-detection/tasks/positive-trigger.yaml +15 -0
- package/evals/eval-harness/eval.yaml +26 -0
- package/evals/eval-harness/tasks/negative-trigger.yaml +16 -0
- package/evals/eval-harness/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-discipline/eval.yaml +26 -0
- package/evals/fable-discipline/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-discipline/tasks/positive-trigger.yaml +15 -0
- package/evals/fable-mode/eval.yaml +26 -0
- package/evals/fable-mode/tasks/negative-tier2-no-model.yaml +18 -0
- package/evals/fable-mode/tasks/negative-trigger.yaml +16 -0
- package/evals/fable-mode/tasks/positive-haiku-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-opus-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-sonnet-mode.yaml +16 -0
- package/evals/fable-mode/tasks/positive-trigger.yaml +15 -0
- package/evals/find-skills/eval.yaml +26 -0
- package/evals/find-skills/tasks/negative-trigger.yaml +16 -0
- package/evals/find-skills/tasks/positive-trigger.yaml +15 -0
- package/evals/git-commit/eval.yaml +26 -0
- package/evals/git-commit/tasks/negative-trigger.yaml +16 -0
- package/evals/git-commit/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-me/eval.yaml +26 -0
- package/evals/grill-me/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-me/tasks/positive-trigger.yaml +15 -0
- package/evals/grill-with-docs/eval.yaml +26 -0
- package/evals/grill-with-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/grill-with-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/harness-everything/eval.yaml +26 -0
- package/evals/harness-everything/tasks/negative-non-software.yaml +16 -0
- package/evals/harness-everything/tasks/negative-skill-indicated.yaml +15 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier2.yaml +16 -0
- package/evals/harness-everything/tasks/positive-tier3.yaml +16 -0
- package/evals/improve-codebase-architecture/eval.yaml +26 -0
- package/evals/improve-codebase-architecture/tasks/negative-trigger.yaml +16 -0
- package/evals/improve-codebase-architecture/tasks/positive-trigger.yaml +15 -0
- package/evals/install-cognitive-os/eval.yaml +26 -0
- package/evals/install-cognitive-os/tasks/negative-trigger.yaml +16 -0
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/eval.yaml +26 -0
- package/evals/multi-agent-workspace/tasks/negative-trigger.yaml +15 -0
- package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +15 -0
- package/evals/repo-docs/eval.yaml +26 -0
- package/evals/repo-docs/tasks/negative-trigger.yaml +16 -0
- package/evals/repo-docs/tasks/positive-trigger.yaml +15 -0
- package/evals/rewrite-commits/eval.yaml +26 -0
- package/evals/rewrite-commits/tasks/negative-trigger.yaml +16 -0
- package/evals/rewrite-commits/tasks/positive-trigger.yaml +15 -0
- package/evals/security-review/eval.yaml +26 -0
- package/evals/security-review/tasks/negative-trigger.yaml +16 -0
- package/evals/security-review/tasks/positive-trigger.yaml +15 -0
- package/evals/self-evolve/eval.yaml +26 -0
- package/evals/self-evolve/tasks/negative-trigger.yaml +16 -0
- package/evals/self-evolve/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-creator/eval.yaml +26 -0
- package/evals/skill-creator/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-creator/tasks/positive-trigger.yaml +15 -0
- package/evals/skill-style/eval.yaml +26 -0
- package/evals/skill-style/tasks/negative-trigger.yaml +16 -0
- package/evals/skill-style/tasks/positive-trigger.yaml +15 -0
- package/evals/tdd/eval.yaml +26 -0
- package/evals/tdd/tasks/negative-trivia.yaml +16 -0
- package/evals/tdd/tasks/positive-bugfix.yaml +15 -0
- package/evals/tdd/tasks/positive-feature.yaml +16 -0
- package/evals/tdd/tasks/positive-refactor.yaml +15 -0
- package/evals/to-spec/eval.yaml +26 -0
- package/evals/to-spec/tasks/negative-trigger.yaml +16 -0
- package/evals/to-spec/tasks/positive-trigger.yaml +15 -0
- package/evals/to-tickets/eval.yaml +26 -0
- package/evals/to-tickets/tasks/negative-trigger.yaml +15 -0
- package/evals/to-tickets/tasks/positive-trigger.yaml +15 -0
- package/evals/todo-driven-workflow/eval.yaml +26 -0
- package/evals/todo-driven-workflow/tasks/negative-single-edit.yaml +16 -0
- package/evals/todo-driven-workflow/tasks/positive-multi-step.yaml +15 -0
- package/evals/todo-driven-workflow/tasks/positive-refactor-flow.yaml +15 -0
- package/evals/using-git-worktrees/eval.yaml +26 -0
- package/evals/using-git-worktrees/tasks/negative-trigger.yaml +15 -0
- package/evals/using-git-worktrees/tasks/positive-trigger.yaml +15 -0
- package/evals/verification-loop/eval.yaml +26 -0
- package/evals/verification-loop/tasks/negative-discussion.yaml +16 -0
- package/evals/verification-loop/tasks/positive-pre-ship.yaml +15 -0
- package/evals/verify-before-claim/eval.yaml +26 -0
- package/evals/verify-before-claim/tasks/negative-trigger.yaml +16 -0
- package/evals/verify-before-claim/tasks/positive-trigger.yaml +15 -0
- package/evals/zoom-out/eval.yaml +26 -0
- package/evals/zoom-out/tasks/negative-trivial-edit.yaml +16 -0
- package/evals/zoom-out/tasks/positive-repeated-failure.yaml +15 -0
- package/fable-discipline/SKILL.md +31 -28
- package/fable-discipline/references/discipline-rules.md +22 -0
- package/fable-mode/CONTRACT-FORMAT.md +51 -51
- package/fable-mode/SKILL.md +31 -43
- package/fable-mode/agents/fable-orchestrator.md +99 -92
- package/fable-mode/agents/fable-verifier.md +32 -32
- package/fable-mode/agents/fable-worker-haiku.md +52 -48
- package/fable-mode/agents/fable-worker-sonnet.md +53 -49
- package/fable-mode/execution-guardrails/SKILL.md +43 -87
- package/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/fable-mode/fable-haiku/SKILL.md +36 -48
- package/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/fable-mode/fable-opus/SKILL.md +36 -0
- package/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/fable-mode/model-matrix.json +32 -0
- package/fable-mode/references/execution-phases.md +37 -0
- package/fable-mode/references/model-matrix.md +39 -0
- package/fable-mode/scripts/model-selector.js +129 -0
- package/find-skills/SKILL.md +48 -0
- package/find-skills/references/discovery-flow.md +102 -0
- package/find-skills/scripts/use-skill.js +106 -0
- package/git-commit/SKILL.md +35 -45
- package/git-commit/guides/ANGULAR_STYLE.md +60 -60
- package/git-commit/guides/COMMIT_GENERATION.md +23 -23
- package/git-commit/guides/LANGUAGE_DETECTION.md +16 -16
- package/git-commit/guides/MAIN_REPO.md +14 -14
- package/git-commit/guides/SUBMODULES.md +27 -27
- package/git-commit/references/commit-flow.md +30 -0
- package/grill-me/SKILL.md +34 -27
- package/grill-me/references/grilling-playbook.md +38 -0
- package/grill-with-docs/ADR-FORMAT.md +47 -47
- package/grill-with-docs/CONTEXT-FORMAT.md +63 -63
- package/grill-with-docs/SKILL.md +33 -91
- package/grill-with-docs/references/session-playbook.md +97 -0
- package/harness-everything/SKILL.md +29 -128
- package/harness-everything/examples/README.md +9 -9
- package/harness-everything/examples/todo-workflow.md +22 -22
- package/harness-everything/references/skill-registry.md +39 -0
- package/harness-everything/references/triage-and-tiers.md +116 -0
- package/harness-everything/scripts/bootstrap.js +203 -140
- package/harness-everything/scripts/routing-keywords.json +320 -0
- package/harness-everything/scripts/self-heal.js +241 -189
- package/harness-everything/scripts/tier-router.js +334 -219
- package/harness-everything/scripts/verify-gate.js +100 -31
- package/hooks/hooks.json +155 -155
- package/hooks/scripts/atomic-commit-check.js +68 -68
- package/hooks/scripts/boundary-guard.js +55 -55
- package/hooks/scripts/context-compact.js +114 -114
- package/hooks/scripts/contract-test.js +79 -79
- package/hooks/scripts/depth-guard.js +65 -65
- package/hooks/scripts/lib/harness-state.js +139 -228
- package/hooks/scripts/lib/platforms/claude.js +72 -19
- package/hooks/scripts/lib/platforms/codex.js +24 -6
- package/hooks/scripts/lib/platforms/continue.js +23 -5
- package/hooks/scripts/lib/platforms/copilot.js +23 -5
- package/hooks/scripts/lib/platforms/cursor.js +23 -5
- package/hooks/scripts/lib/platforms/hermes.js +57 -57
- package/hooks/scripts/lib/platforms/index.js +17 -17
- package/hooks/scripts/lib/platforms/worktrees.js +20 -20
- package/hooks/scripts/reset-circuit-breaker.js +30 -30
- package/hooks/scripts/rule-of-3-tracker.js +124 -91
- package/hooks/scripts/rule-of-3.js +187 -176
- package/hooks/scripts/state-persist.js +99 -99
- package/hooks/scripts/stop-gate.js +80 -80
- package/hooks/scripts/subagent-scope-guard.js +88 -88
- package/improve-codebase-architecture/SKILL.md +31 -35
- package/improve-codebase-architecture/guides/DEEPENING.md +37 -37
- package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -123
- package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -44
- package/improve-codebase-architecture/guides/LANGUAGE.md +53 -53
- package/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/install-cognitive-os/SKILL.md +23 -80
- package/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/multi-agent-workspace/SKILL.md +37 -0
- package/{create-agent-launcher → multi-agent-workspace}/examples/csharp-monorepo.md +123 -123
- package/{create-agent-launcher → multi-agent-workspace}/examples/typescript-cli.md +126 -126
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-claude.md +77 -77
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-codex.md +146 -146
- package/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-copilot.md +74 -74
- package/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/{create-agent-launcher → multi-agent-workspace}/guidelines/role-capabilities.md +108 -108
- package/multi-agent-workspace/references/agency-agents.md +31 -0
- package/multi-agent-workspace/references/architecture-guide.md +32 -0
- package/multi-agent-workspace/references/orchestration.md +48 -0
- package/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/multi-agent-workspace/scripts/index_memory.js +51 -0
- package/multi-agent-workspace/scripts/scaffold.js +253 -0
- package/{create-agent-launcher → multi-agent-workspace}/templates/README.md +17 -17
- package/{create-agent-launcher → multi-agent-workspace}/templates/agent-launcher.md +67 -67
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-csharp.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-polyglot.md +61 -61
- package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-ts.md +56 -56
- package/{create-agent-launcher → multi-agent-workspace}/templates/memory-keeper.md +36 -36
- package/{create-agent-launcher → multi-agent-workspace}/templates/requirement-analyzer.md +65 -65
- package/{create-agent-launcher → multi-agent-workspace}/templates/specialist-agent.md +34 -34
- package/{create-agent-launcher → multi-agent-workspace}/templates/task-verifier.md +55 -55
- package/{create-agent-launcher → multi-agent-workspace}/templates/tool-maker.md +32 -32
- package/multi-agent-workspace/workflows/01-init.md +22 -0
- package/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/multi-agent-workspace/workflows/03-generation.md +27 -0
- package/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/opencode-plugin/README.md +94 -0
- package/opencode-plugin/hooks/circuit-breaker.js +121 -0
- package/opencode-plugin/hooks/compliance.js +128 -0
- package/opencode-plugin/hooks/post-edit.js +62 -0
- package/opencode-plugin/hooks/pre-complete.js +65 -0
- package/opencode-plugin/hooks/verify.js +140 -0
- package/opencode-plugin/plugin.json +17 -0
- package/opencode-plugin/test-hook-firing.js +198 -0
- package/package.json +50 -22
- package/references/definition-of-done.md +67 -0
- package/references/performance-checklist.md +153 -0
- package/references/platform-advisory-template.md +58 -0
- package/references/security-checklist.md +205 -0
- package/repo-docs/SKILL.md +38 -37
- package/repo-docs/references/process-guide.md +60 -0
- package/repo-docs/templates/agents-template.md +264 -264
- package/repo-docs/templates/knowledge-agents-template.md +50 -50
- package/repo-docs/templates/knowledge-readme-template.md +83 -83
- package/repo-docs/templates/manual-recon.md +117 -117
- package/repo-docs/templates/multi-skills-readme-template.md +72 -72
- package/repo-docs/templates/product-agents-template.md +78 -78
- package/repo-docs/templates/product-readme-template.md +95 -95
- package/repo-docs/templates/readme-template.md +136 -136
- package/repo-docs/templates/single-skill-readme-template.md +70 -70
- package/repo-docs/templates/skills-agents-template.md +60 -60
- package/rewrite-commits/SKILL.md +22 -17
- package/rewrite-commits/references/workflow.md +18 -0
- package/scripts/installer.js +546 -506
- package/scripts/lib/advisory-text.js +8 -4
- package/scripts/lib/claude-hooks.js +18 -5
- package/scripts/lib/gitignore.js +199 -80
- package/scripts/lib/manifest.js +68 -4
- package/scripts/lib/prompts.js +1 -1
- package/scripts/lib/skills.js +87 -1
- package/scripts/verify-install.js +147 -0
- package/security-review/SKILL.md +35 -496
- package/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/security-review/references/security-checklist.md +478 -0
- package/security-review/scripts/audit-secrets.js +84 -0
- package/self-evolve/SKILL.md +38 -45
- package/self-evolve/references/memory-resolution.md +58 -0
- package/self-evolve/scripts/persist-memory.js +139 -40
- package/self-evolve/scripts/register-dynamic-skill.js +251 -0
- package/self-evolve/scripts/self-regression.js +110 -113
- package/skill-creator/SKILL.md +34 -76
- package/skill-creator/references/authoring-workflow.md +14 -0
- package/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/skill-creator/references/quality-checklist.md +15 -0
- package/skill-creator/references/quality-principles.md +81 -81
- package/skill-creator/references/testing-workflow.md +27 -32
- package/skill-style/SKILL.md +33 -39
- package/skill-style/references/style-guide.md +36 -0
- package/tdd/SKILL.md +34 -43
- package/tdd/guides/deep-modules.md +33 -33
- package/tdd/guides/interface-design.md +31 -31
- package/tdd/guides/mocking.md +59 -59
- package/tdd/guides/refactoring.md +10 -10
- package/tdd/guides/tests.md +61 -61
- package/tdd/references/core-discipline.md +49 -0
- package/to-spec/SKILL.md +42 -0
- package/to-spec/references/process.md +77 -0
- package/to-spec/scripts/check-project-docs.js +259 -0
- package/to-spec/templates/cli-reference.md +31 -0
- package/to-spec/templates/dev-doc.md +27 -0
- package/to-spec/templates/feature-spec.md +59 -0
- package/to-spec/templates/schema-doc.md +31 -0
- package/to-tickets/SKILL.md +42 -0
- package/to-tickets/references/publishing-and-templates.md +97 -0
- package/todo-driven-workflow/SKILL.md +27 -41
- package/todo-driven-workflow/references/execution-guide.md +27 -0
- package/using-git-worktrees/SKILL.md +29 -141
- package/using-git-worktrees/references/workflow-details.md +108 -0
- package/verification-loop/SKILL.md +29 -121
- package/verification-loop/references/verification-phases.md +116 -0
- package/verification-loop/templates/verification-report.template.md +15 -0
- package/verify-before-claim/SKILL.md +27 -70
- package/verify-before-claim/references/verification-guide.md +73 -0
- package/zoom-out/SKILL.md +28 -76
- package/zoom-out/references/circuit-breaker.md +66 -0
- package/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/build-multi-agent-system/AGENTS.md +0 -50
- package/build-multi-agent-system/README.md +0 -63
- package/build-multi-agent-system/SKILL.md +0 -68
- package/create-agent-launcher/SKILL.md +0 -48
- package/create-agent-launcher/guidelines/platform-gemini.md +0 -101
- package/create-agent-launcher/workflows/01-init.md +0 -22
- package/create-agent-launcher/workflows/02-analysis.md +0 -26
- package/create-agent-launcher/workflows/03-generation.md +0 -29
- package/create-agent-launcher/workflows/04-launcher.md +0 -20
- package/docs/workflows/build-multi-agent-system.md +0 -62
- package/docs/workflows/create-agent-launcher.md +0 -62
- package/eval-framework/behavioral-test.js +0 -105
- package/eval-framework/mechanism-test.js +0 -251
- package/harness-everything/scripts/todo-cli.js +0 -157
- package/skill-style/STYLE.md +0 -37
|
@@ -1,113 +1,110 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
/**
|
|
3
|
-
* Harness Self-Regression Test Suite
|
|
4
|
-
*
|
|
5
|
-
*
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
const
|
|
10
|
-
const
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
console.log(
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
if (
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
console.
|
|
53
|
-
|
|
54
|
-
}
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
}
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
const
|
|
89
|
-
if (
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
}
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
}
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
console.error(
|
|
106
|
-
console.log(
|
|
107
|
-
process.exit(1);
|
|
108
|
-
}
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
console.log("=================================================");
|
|
112
|
-
process.exit(0);
|
|
113
|
-
}
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Harness Self-Regression Test Suite.
|
|
4
|
+
*
|
|
5
|
+
* This is the free, deterministic gate. Model sessions remain on-demand;
|
|
6
|
+
* their results must not be confused with a local tracker simulation.
|
|
7
|
+
*/
|
|
8
|
+
|
|
9
|
+
const { spawnSync } = require('child_process');
|
|
10
|
+
const path = require('path');
|
|
11
|
+
const fs = require('fs');
|
|
12
|
+
|
|
13
|
+
const projectRoot = path.resolve(__dirname, '..', '..');
|
|
14
|
+
let hasErrors = false;
|
|
15
|
+
|
|
16
|
+
console.log('=================================================');
|
|
17
|
+
console.log(' Harness OS - Self-Regression Test Suite ');
|
|
18
|
+
console.log('=================================================');
|
|
19
|
+
|
|
20
|
+
function runNode(label, script, args = [], options = {}) {
|
|
21
|
+
const result = spawnSync('node', [script, ...args], {
|
|
22
|
+
cwd: projectRoot,
|
|
23
|
+
stdio: 'inherit',
|
|
24
|
+
...options,
|
|
25
|
+
});
|
|
26
|
+
if (result.status !== 0) {
|
|
27
|
+
console.error(` FAIL ${label}`);
|
|
28
|
+
hasErrors = true;
|
|
29
|
+
} else {
|
|
30
|
+
console.log(` PASS ${label}`);
|
|
31
|
+
}
|
|
32
|
+
return result;
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
// 1. Syntax-check every shipped JavaScript file.
|
|
36
|
+
console.log('\n[Phase 1] Static Syntax Check...');
|
|
37
|
+
const foldersToScan = ['harness-everything', 'hooks', 'environment-detection', 'self-evolve', 'ci', 'eval-harness', 'scripts', 'bin', 'to-spec', 'to-tickets', 'find-skills', 'fable-mode'];
|
|
38
|
+
const jsFiles = [];
|
|
39
|
+
function walkDir(dir) {
|
|
40
|
+
if (!fs.existsSync(dir)) return;
|
|
41
|
+
for (const file of fs.readdirSync(dir)) {
|
|
42
|
+
const filePath = path.join(dir, file);
|
|
43
|
+
if (fs.statSync(filePath).isDirectory()) walkDir(filePath);
|
|
44
|
+
else if (filePath.endsWith('.js')) jsFiles.push(filePath);
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
for (const folder of foldersToScan) walkDir(path.join(projectRoot, folder));
|
|
48
|
+
for (const file of jsFiles) {
|
|
49
|
+
const check = spawnSync('node', ['--check', file]);
|
|
50
|
+
if (check.status !== 0) {
|
|
51
|
+
console.error(` FAIL syntax: ${path.relative(projectRoot, file)}`);
|
|
52
|
+
console.error(check.stderr.toString().trim());
|
|
53
|
+
hasErrors = true;
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
console.log(` PASS syntax: ${jsFiles.length} JavaScript file(s)`);
|
|
57
|
+
|
|
58
|
+
// 1b. Smoke-test the public CLI wrappers.
|
|
59
|
+
console.log('\n[Phase 1b] CLI Command Smoke Test...');
|
|
60
|
+
const cliPath = path.join(projectRoot, 'bin', 'cli.js');
|
|
61
|
+
const nextCheck = spawnSync('node', [cliPath, 'next', 'add a new login endpoint with tests'], { cwd: projectRoot });
|
|
62
|
+
if (nextCheck.status !== 0 || !nextCheck.stdout.toString().includes('RECOMMENDED TIER')) {
|
|
63
|
+
console.error(' FAIL `harness next` did not produce a routing recommendation.');
|
|
64
|
+
hasErrors = true;
|
|
65
|
+
} else console.log(' PASS `harness next` produced a routing recommendation.');
|
|
66
|
+
const verifyCheck = spawnSync('node', [cliPath, 'verify'], {
|
|
67
|
+
cwd: projectRoot,
|
|
68
|
+
env: { ...process.env, HARNESS_SKIP_PROJECT_CHECKS: '1' },
|
|
69
|
+
});
|
|
70
|
+
if (verifyCheck.status !== 0) {
|
|
71
|
+
console.error(' FAIL `harness verify` exited non-zero on a skipped check.');
|
|
72
|
+
hasErrors = true;
|
|
73
|
+
} else console.log(' PASS `harness verify` exited 0 on a skipped check.');
|
|
74
|
+
|
|
75
|
+
// 2. Deterministic routing matrix.
|
|
76
|
+
console.log('\n[Phase 2] Routing Verification Check...');
|
|
77
|
+
runNode('routing matrix', path.join(projectRoot, 'ci', 'runner.js'));
|
|
78
|
+
runNode('skill route coverage', path.join(projectRoot, 'ci', 'skill-routing-check.js'));
|
|
79
|
+
|
|
80
|
+
// 3. Static integrity gates; behavioral evals are validated, not executed.
|
|
81
|
+
console.log('\n[Phase 3] Skill Reference and Behavioral Case Checks...');
|
|
82
|
+
runNode('skill reference check', path.join(projectRoot, 'ci', 'reference-check.js'));
|
|
83
|
+
runNode('behavioral case validation', path.join(projectRoot, 'behavioral-evals', 'run.js'), ['validate']);
|
|
84
|
+
// 3b. Run Fable model mode contract tests (selector + router separation)
|
|
85
|
+
console.log('\n[Phase 3b] Fable Model Mode Contract Tests...');
|
|
86
|
+
const fableModeTestPath = path.join(projectRoot, 'ci', 'fable-mode-test.js');
|
|
87
|
+
if (fs.existsSync(fableModeTestPath)) {
|
|
88
|
+
const fableModeCheck = spawnSync('node', [fableModeTestPath], { stdio: 'inherit', cwd: projectRoot });
|
|
89
|
+
if (fableModeCheck.status !== 0) {
|
|
90
|
+
console.error("\n ❌ Fable model mode contract suite failed!");
|
|
91
|
+
hasErrors = true;
|
|
92
|
+
} else {
|
|
93
|
+
console.log("\n ✅ Fable model mode contract suite 100% Passed!");
|
|
94
|
+
}
|
|
95
|
+
} else {
|
|
96
|
+
console.warn(" ⚠️ ci/fable-mode-test.js not found. Skipping Phase 3.");
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
// 4. Hook/mechanism checks.
|
|
100
|
+
console.log('\n[Phase 4] Mechanism Test Suite (Claude Code hooks)...');
|
|
101
|
+
runNode('mechanism suite', path.join(projectRoot, 'ci', 'mechanism-test.js'));
|
|
102
|
+
|
|
103
|
+
console.log('\n=================================================');
|
|
104
|
+
if (hasErrors) {
|
|
105
|
+
console.error(' FAIL: self-regression found one or more problems.');
|
|
106
|
+
console.log('=================================================');
|
|
107
|
+
process.exit(1);
|
|
108
|
+
}
|
|
109
|
+
console.log(' PASS: all deterministic self-regression checks passed.');
|
|
110
|
+
console.log('=================================================');
|
package/skill-creator/SKILL.md
CHANGED
|
@@ -1,91 +1,49 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: skill-creator
|
|
3
|
-
description: Author, audit, and refactor SKILL.md files against
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
description: Author, audit, and refactor SKILL.md files against one quality bar — the Skill Contract format. Use when creating a new skill, refactoring a SKILL.md, checking overlap, or packaging an insight as a dynamic skill.
|
|
4
|
+
license: Apache-2.0
|
|
5
|
+
metadata:
|
|
6
|
+
author: Miya Daniel
|
|
7
|
+
version: 0.3.6
|
|
6
8
|
---
|
|
7
9
|
|
|
8
|
-
# Skill Creator
|
|
10
|
+
# Skill Creator
|
|
9
11
|
|
|
10
|
-
|
|
12
|
+
SKILL.md authoring and audit.
|
|
11
13
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
| **State Mutations** | Writes/edits `<skill>/SKILL.md` (+ optional `guides/`/`references/`); for new static skills, adds one row to `harness-everything/SKILL.md` §5 and, if keyword-routable, one line to `tier-router.js`'s `recommendedGuides`. |
|
|
17
|
-
| **Enforcement Gate** | A new or edited `SKILL.md` **MUST NOT** be registered or persisted until every item in the Quality Checklist (§3) is checked off — for dynamic skills this is self-evolve's actual persistence gate, not optional polish. |
|
|
18
|
-
|
|
19
|
-
`skill-style/SKILL.md` defines *the shape* a SKILL.md must take (the Skill Contract table, the imperative tone) but not *how to get there well* — how to size a skill, when to push detail into a reference file, how to tell a real enforcement gate from a restated default, or where a rule belongs before it starts drifting across three files. Those questions come from two sources studied for this skill: [`writing-great-skills`](../mattpocock-skills/skills/productivity/writing-great-skills/SKILL.md) and Anthropic's own `skill-creator`. Neither is followed blindly — §1 says where Harness's own conventions override them, and why.
|
|
20
|
-
|
|
21
|
-
## 1. Two philosophies, reconciled
|
|
22
|
-
|
|
23
|
-
Harness gets predictability from a **physical** mechanism: the Skill Contract table's Enforcement Gate row, backed by a script that exits non-zero. External skill-writing research gets predictability from a **cognitive** mechanism: minimal context load, leading words, explaining *why* instead of just forbidding. Both are real and both are validated — see the 2026-07-22 skill quality audit for the evidence — and they don't actually conflict where it counts:
|
|
24
|
-
|
|
25
|
-
- **Keep** the Skill Contract table and `MUST`/`PROHIBITED` for anything with a real Enforcement Gate: an irreversible action, a script that can exit non-zero, a state mutation another skill depends on. That's a hard guardrail; state it as one.
|
|
26
|
-
- **Don't** reach for `MUST`/`PROHIBITED` as the default tone for style preferences with no gate behind them — "don't write long paragraphs" doesn't need to be a `MUST NOT`; say what good output looks like instead. Every imperative not backed by an actual gate is a candidate for rewriting positively.
|
|
27
|
-
- **Always** apply the parts with no philosophical conflict at all: don't duplicate a meaning across files, don't restate what the model already does by default, push detail behind a pointer once a skill grows past what every reader needs inline, and reuse this repo's own leading words (`Tier 1/2/3`, `circuit breaker`/`zoom out`, `Rule of 3`, `Red/Green/Refactor`) instead of coining new ones for the same concept.
|
|
28
|
-
|
|
29
|
-
Full glossary and worked examples from both sources: `references/quality-principles.md`.
|
|
14
|
+
## USE FOR:
|
|
15
|
+
- Create a skill from scratch
|
|
16
|
+
- Audit/refactor a SKILL.md; check overlap between skills
|
|
17
|
+
- Packaging a session insight as a dynamic skill (self-evolve)
|
|
30
18
|
|
|
31
|
-
##
|
|
19
|
+
## DO NOT USE FOR:
|
|
20
|
+
- Project docs that are not skills — instead use `repo-docs` / `to-spec`
|
|
21
|
+
- Third-party skill installs (find-skills)
|
|
22
|
+
- Code style outside SKILL.md files
|
|
32
23
|
|
|
33
|
-
|
|
34
|
-
Answer these, pulling from the conversation first if it already has the answers:
|
|
35
|
-
1. What should this skill make the agent do that it wouldn't do by default?
|
|
36
|
-
2. What's the *one* canonical sentence describing when it fires? This sentence becomes the frontmatter `description` — and nothing else. Don't let the registry row or `tier-router.js` invent independent wording later: the audit report §1.2 found the same skill carrying two different one-line descriptions inside `tier-router.js` itself, because nobody treated the frontmatter line as the source of truth.
|
|
37
|
-
3. Which Tier does it belong to, or is it an always-on discipline / meta skill outside the tier system?
|
|
38
|
-
4. Does an existing skill already own this ground? Grep `harness-everything/SKILL.md` §5 first — a near-duplicate skill is worse than a slightly-too-broad existing one.
|
|
24
|
+
## Skill Contract
|
|
39
25
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
### Step 3 — Test against real prompts
|
|
47
|
-
Don't ship on vibes alone. Write 2-3 prompts a real user would actually type — not "test the skill," the kind of message that should trigger it. Using this repo's own `create-agent-launcher`, spawn one subagent *with* the draft skill loaded and one *without* (baseline), same prompt, and compare. This is the same with/without comparison Anthropic's `skill-creator` runs through a Python eval-viewer — Harness doesn't need a second toolchain for it; `create-agent-launcher` already does resource-isolated subagent spawning. Read both transcripts, not just final outputs — repeated dead-end exploration in the with-skill transcript is a sign the skill under-specifies something, not that the model is weak.
|
|
48
|
-
|
|
49
|
-
Full adapted testing loop — when to bother, what to look for in transcripts, how a router-based system checks trigger phrasing without a native autonomous picker: `references/testing-workflow.md`.
|
|
50
|
-
|
|
51
|
-
### Step 4 — Prune, then register
|
|
52
|
-
Run the Quality Checklist (§3). Then:
|
|
53
|
-
- **Static skill**: add one row to `harness-everything/SKILL.md` §5 (Layer + "Activated when," quoting the frontmatter description rather than rewriting it), and if the trigger is keyword-detectable, one line to the matching keyword block in `harness-everything/scripts/tier-router.js`.
|
|
54
|
-
- **Dynamic skill** (from `self-evolve`): follow §4 instead — different location, different lifecycle, same checklist.
|
|
55
|
-
|
|
56
|
-
## 3. Quality Checklist (the actual gate)
|
|
57
|
-
|
|
58
|
-
A skill isn't done until every line here is true. This is deliberately a flat checklist, not steps — check them in any order, but check all of them.
|
|
26
|
+
| Component | Specification |
|
|
27
|
+
| :--- | :--- |
|
|
28
|
+
| **Trigger / Input** | New skill; SKILL.md audit; self-evolve packaging. |
|
|
29
|
+
| **Expected Output** | SKILL.md passing the Quality Checklist. |
|
|
30
|
+
| **State Mutations** | Writes `<skill>/SKILL.md`; updates registry/generated folder. |
|
|
31
|
+
| **Enforcement Gate** | Quality Checklist complete before registering. |
|
|
59
32
|
|
|
60
|
-
|
|
61
|
-
- [ ] **Single source of truth**: the frontmatter `description` is the *only* place this skill's purpose is originally written. The registry row and any `tier-router.js` line quote or tightly paraphrase it — they don't independently re-describe the skill.
|
|
62
|
-
- [ ] **No block over ~5 lines duplicated verbatim elsewhere** — in this file or in another always/near-always-loaded skill. (`install-cognitive-os` vs. `harness-everything`'s 33-line ADHD-output block, flagged in the audit report, is the canonical counter-example — don't add a second one.)
|
|
63
|
-
- [ ] **Every `MUST`/`MUST NOT`/`PROHIBITED` has a real gate behind it**, or is rewritten as a positive statement of the target behavior. A prohibition with no gate and no "why" sentence next to it is a smell, not a strength.
|
|
64
|
-
- [ ] **No no-op lines** — for each imperative sentence, ask whether it changes behavior versus what the model already does by default. ("Be thorough" reads as a no-op; a concrete completion criterion or a stronger leading word doesn't.)
|
|
65
|
-
- [ ] **Body fits the information hierarchy** — steps for ordered action, flat reference for rules, detail past what every branch needs pushed to `guides/`/`references/` with a "when to open this" pointer.
|
|
66
|
-
- [ ] **Reuses existing leading words** (`Tier 1/2/3`, `circuit breaker`/`zoom out`, `Rule of 3`, `Red/Green/Refactor`, `Discover > Think > Try > Summarize > Record`) instead of naming a new concept for something this repo already has a word for.
|
|
67
|
-
- [ ] **Completion criteria are checkable** — tied to an exit code, a file's existence, a grep result, or explicit human confirmation, not "when it feels done."
|
|
68
|
-
- [ ] **Description reads as a human-facing sentence**, not just a trigger-phrase list — `scripts/installer.js`'s interactive picker prints it next to a checkbox for a person deciding whether to install this skill; that's its real first reader in this architecture (audit report §2.1).
|
|
33
|
+
## Authoring Workflow
|
|
69
34
|
|
|
70
|
-
|
|
35
|
+
1. Grep the registry for near-duplicates; the when-to-fire sentence becomes the frontmatter description.
|
|
36
|
+
2. Draft the Contract table first (forces the Enforcement Gate), then steps/flat reference; push branch-only detail to references/.
|
|
37
|
+
3. A/B-test via `multi-agent-workspace` subagents; read both transcripts.
|
|
38
|
+
4. Run the Quality Checklist; register (static row quotes the description; dynamic below).
|
|
71
39
|
|
|
72
|
-
|
|
40
|
+
## Dynamic Skill Generation Contract
|
|
73
41
|
|
|
74
|
-
|
|
75
|
-
- **Required frontmatter**, in addition to `name`/`description`:
|
|
76
|
-
```yaml
|
|
77
|
-
metadata:
|
|
78
|
-
type: dynamic
|
|
79
|
-
generated: <YYYY-MM-DD>
|
|
80
|
-
source: <one-line pointer to the session/root-cause that produced it, e.g. "zoom-out recovery, 2026-07-22, ORM connection-pool exhaustion">
|
|
81
|
-
status: draft
|
|
82
|
-
```
|
|
83
|
-
- **Gate**: run the Quality Checklist (§3) before the skill file is written. A dynamic skill skips human PR review by design, so this checklist is the only review it gets — don't skip it because the insight "feels obviously right" in the moment; that's exactly the state self-evolve is triggered from.
|
|
84
|
-
- **Lifecycle**: `status: draft` at birth. Promote to `status: active` once it fires successfully in a *different* task than the one that produced it — a skill that has only proven itself on the bug that spawned it hasn't proven generality yet. If a dynamic skill goes unused or stops matching reality, mark `status: deprecated` rather than deleting it silently — deleting is fine once something else supersedes it, but a silent disappearance is harder to debug later than a stale-but-labeled file.
|
|
85
|
-
- **Promotion to static**: once a dynamic skill has proven itself general — used successfully across genuinely unrelated tasks, not repeated instances of the same bug — promote it: move it to the repo root, drop the `metadata.type: dynamic` block, and register it in `harness-everything/SKILL.md` §5 like any other skill. At that point it's subject to the same review/PR discipline as everything else here.
|
|
42
|
+
Only generalizable multi-step procedures or enforcement contracts qualify as skills; simple constraints go to `memories/repo/RULES.md`.
|
|
86
43
|
|
|
87
|
-
|
|
44
|
+
- Location: `.claude/harness-everything/skills/generated/<kebab-case-name>/SKILL.md`
|
|
45
|
+
- Frontmatter beyond name/description: triggers (3-6 keywords), metadata type: dynamic, generated, source, status: draft.
|
|
46
|
+
- Gate: Quality Checklist first.
|
|
47
|
+
- Lifecycle: draft → active after firing on a different task; deprecate rather than silently delete; promote once proven general.
|
|
88
48
|
|
|
89
|
-
|
|
90
|
-
- `create-agent-launcher/SKILL.md` — used in §2 Step 3 to spawn with/without-skill test subagents.
|
|
91
|
-
- `self-evolve/SKILL.md` — the caller for §4; it loads this skill rather than duplicating the Dynamic Skill Generation Contract inline.
|
|
49
|
+
Deep dive: references/{authoring-workflow,dynamic-generation-contract,quality-checklist}.md
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Authoring Workflow Detail
|
|
2
|
+
|
|
3
|
+
Keep the Contract table as the source of truth for input, output, state, and
|
|
4
|
+
the enforcement gate. Use short ordered steps for actions and move branch-only
|
|
5
|
+
detail into references or guides.
|
|
6
|
+
|
|
7
|
+
Before registering a skill, check the Harness registry for overlap, define one
|
|
8
|
+
human-readable frontmatter description, run realistic with/without tests using
|
|
9
|
+
`multi-agent-workspace`, and complete the Quality Checklist. Static skills
|
|
10
|
+
belong in the root catalog and need a routing eval; dynamic skills use the
|
|
11
|
+
generated manifest flow from `self-evolve`.
|
|
12
|
+
|
|
13
|
+
The canonical multi-agent entrypoint is `multi-agent-workspace`; it owns
|
|
14
|
+
bounded delegation and workspace isolation.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Dynamic Skill Generation Contract (for self-evolve)
|
|
2
|
+
|
|
3
|
+
## 4. Dynamic Skill Generation Contract (for `self-evolve`)
|
|
4
|
+
|
|
5
|
+
`self-evolve`'s Step 3 optionally packages a hard-won session insight into a new, standalone skill. This process **MUST** only occur if the LLM has judged that the lesson warrants an independent, complex structural skill rather than a simple memory rule:
|
|
6
|
+
|
|
7
|
+
- **The Judgment Criterion**: Simple constraints or localized tips **MUST** only be written to `memories/repo/RULES.md`. Only highly generalizable, multi-step procedures or custom enforcement contracts (e.g. transaction pooling patterns) should be packaged as a dynamic skill.
|
|
8
|
+
- **Location**: `.claude/harness-everything/skills/generated/<kebab-case-name>/SKILL.md` — not the repo root. `.claude/harness-everything/skills` is already a scope the installer recognizes (`scripts/lib/skills.js`'s `getInstalledSkills`), which keeps dynamically-generated skills discoverable without mixing them into the reviewed, static skill set at the repo root. `generated/` is deliberately excluded from the installer's manifest-tracked, bulk-removable skill set — these are the user's own accumulated learning, not package content, so "uninstall skills" must never sweep them.
|
|
9
|
+
- **Required frontmatter**, in addition to `name`/`description`:
|
|
10
|
+
```yaml
|
|
11
|
+
triggers: [<3-6 specific keywords a real prompt would contain, EN and/or 中文 — not generic words like "error" or "using">]
|
|
12
|
+
metadata:
|
|
13
|
+
type: dynamic
|
|
14
|
+
generated: <YYYY-MM-DD>
|
|
15
|
+
source: <one-line pointer to the session/root-cause that produced it, e.g. "zoom-out recovery, 2026-07-22, ORM connection-pool exhaustion">
|
|
16
|
+
status: draft
|
|
17
|
+
```
|
|
18
|
+
`triggers` is what `tier-router.js` matches against to auto-surface this skill in future sessions — pick words specific enough to the actual problem domain (e.g. `connection-pool`, `transaction-batching`), not words generic enough to fire on unrelated prompts. Omitting this field falls back to `register-dynamic-skill.js`'s naive keyword inference from the name/description, which is intentionally conservative and a worse signal than a deliberate list — don't rely on it.
|
|
19
|
+
- **Gate**: run the Quality Checklist (§3) before the skill file is written. A dynamic skill skips human PR review by design, so this checklist is the only review it gets — don't skip it because the insight "feels obviously right" in the moment; that's exactly the state self-evolve is triggered from.
|
|
20
|
+
- **Lifecycle**: `status: draft` at birth. Promote to `status: active` once it fires successfully in a *different* task than the one that produced it — a skill that has only proven itself on the bug that spawned it hasn't proven generality yet. If a dynamic skill goes unused or stops matching reality, mark `status: deprecated` rather than deleting it silently — deleting is fine once something else supersedes it, but a silent disappearance is harder to debug later than a stale-but-labeled file.
|
|
21
|
+
- **Promotion to static**: once a dynamic skill has proven itself general — used successfully across genuinely unrelated tasks, not repeated instances of the same bug — promote it: move it to the repo root, drop the `metadata.type: dynamic` block, and register it in `harness-everything/SKILL.md` §5 like any other skill. At that point it's subject to the same review/PR discipline as everything else here.
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Quality Checklist (the actual gate)
|
|
2
|
+
|
|
3
|
+
## 3. Quality Checklist (the actual gate)
|
|
4
|
+
|
|
5
|
+
A skill isn't done until every line here is true. This is deliberately a flat checklist, not steps — check them in any order, but check all of them.
|
|
6
|
+
|
|
7
|
+
- [ ] **Skill Contract table present and accurate** — all four rows; Enforcement Gate names a real script/exit-code or an explicit "None" if the skill is pure reference.
|
|
8
|
+
- [ ] **Single source of truth**: the frontmatter `description` is the *only* place this skill's purpose is originally written. The registry row and any `tier-router.js` line quote or tightly paraphrase it — they don't independently re-describe the skill.
|
|
9
|
+
- [ ] **No block over ~5 lines duplicated verbatim elsewhere** — in this file or in another always/near-always-loaded skill. (`install-cognitive-os` vs. `harness-everything`'s 33-line ADHD-output block, flagged in the audit report, is the canonical counter-example — don't add a second one.)
|
|
10
|
+
- [ ] **Every `MUST`/`MUST NOT`/`PROHIBITED` has a real gate behind it**, or is rewritten as a positive statement of the target behavior. A prohibition with no gate and no "why" sentence next to it is a smell, not a strength.
|
|
11
|
+
- [ ] **No no-op lines** — for each imperative sentence, ask whether it changes behavior versus what the model already does by default. ("Be thorough" reads as a no-op; a concrete completion criterion or a stronger leading word doesn't.)
|
|
12
|
+
- [ ] **Body fits the information hierarchy** — steps for ordered action, flat reference for rules, detail past what every branch needs pushed to `guides/`/`references/` with a "when to open this" pointer.
|
|
13
|
+
- [ ] **Reuses existing leading words** (`Tier 1/2/3`, `circuit breaker`/`zoom out`, `Rule of 3`, `Red/Green/Refactor`, `Discover > Think > Try > Summarize > Record`) instead of naming a new concept for something this repo already has a word for.
|
|
14
|
+
- [ ] **Completion criteria are checkable** — tied to an exit code, a file's existence, a grep result, or explicit human confirmation, not "when it feels done."
|
|
15
|
+
- [ ] **Description reads as a human-facing sentence**, not just a trigger-phrase list — `scripts/installer.js`'s interactive picker prints it next to a checkbox for a person deciding whether to install this skill; that's its real first reader in this architecture (audit report §2.1).
|