harness-everything 0.2.2 → 0.3.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (496) hide show
  1. package/.claude-plugin/marketplace.json +27 -0
  2. package/.claude-plugin/plugin.json +51 -0
  3. package/.gitattributes +1 -0
  4. package/.github/workflows/behavioral-evals.yml +36 -0
  5. package/.github/workflows/ci.yml +117 -0
  6. package/.github/workflows/release.yml +45 -0
  7. package/.husky/commit-msg +1 -0
  8. package/.waza.yaml +22 -0
  9. package/AGENTS.md +60 -2
  10. package/BENCHMARK_SOP.md +62 -62
  11. package/CHANGELOG.md +297 -114
  12. package/CODE_OF_CONDUCT.md +24 -0
  13. package/CONTRIBUTING.md +47 -0
  14. package/Dockerfile +18 -18
  15. package/LICENSE +201 -201
  16. package/README.md +279 -197
  17. package/SECURITY.md +33 -0
  18. package/VERIFICATION.md +394 -347
  19. package/ab-test-deep-analysis-2026-08-27.md +383 -0
  20. package/ab-test-results-2026-08-27.md +152 -0
  21. package/ab-test-statistical-analysis-31cases.md +193 -0
  22. package/behavioral-evals/README.md +85 -0
  23. package/behavioral-evals/analyze-plugin-results.js +110 -0
  24. package/behavioral-evals/cases/baseline-add-feature.yaml +34 -0
  25. package/behavioral-evals/cases/baseline-api-design.yaml +32 -0
  26. package/behavioral-evals/cases/baseline-debugging.yaml +31 -0
  27. package/behavioral-evals/cases/baseline-dependency.yaml +24 -0
  28. package/behavioral-evals/cases/baseline-documentation.yaml +34 -0
  29. package/behavioral-evals/cases/baseline-error-handling.yaml +31 -0
  30. package/behavioral-evals/cases/baseline-git-commit.yaml +33 -0
  31. package/behavioral-evals/cases/baseline-performance-monitoring.yaml +28 -0
  32. package/behavioral-evals/cases/baseline-performance.yaml +34 -0
  33. package/behavioral-evals/cases/baseline-refactor-with-tests.yaml +35 -0
  34. package/behavioral-evals/cases/baseline-refactor.yaml +27 -0
  35. package/behavioral-evals/cases/baseline-security-review.yaml +28 -0
  36. package/behavioral-evals/cases/baseline-simple-bugfix.yaml +33 -0
  37. package/behavioral-evals/cases/baseline-tdd-workflow.yaml +28 -0
  38. package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +50 -0
  39. package/behavioral-evals/cases/pressure-budget-skip-review.yaml +27 -0
  40. package/behavioral-evals/cases/pressure-bypass-review.yaml +27 -0
  41. package/behavioral-evals/cases/pressure-complexity-skip.yaml +37 -0
  42. package/behavioral-evals/cases/pressure-expert-skip-testing.yaml +31 -0
  43. package/behavioral-evals/cases/pressure-fatigue-skip.yaml +33 -0
  44. package/behavioral-evals/cases/pressure-management-skip-all.yaml +28 -0
  45. package/behavioral-evals/cases/pressure-scope-bypass.yaml +28 -0
  46. package/behavioral-evals/cases/pressure-skip-docs.yaml +50 -0
  47. package/behavioral-evals/cases/pressure-skip-error-handling.yaml +35 -0
  48. package/behavioral-evals/cases/pressure-skip-security.yaml +32 -0
  49. package/behavioral-evals/cases/pressure-skip-tests.yaml +27 -0
  50. package/behavioral-evals/cases/pressure-skip-verification.yaml +43 -0
  51. package/behavioral-evals/cases/pressure-social-skip-review.yaml +28 -0
  52. package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +47 -0
  53. package/behavioral-evals/cases/scope-discipline.yaml +32 -0
  54. package/behavioral-evals/cases/tier1-no-overplan.yaml +26 -0
  55. package/behavioral-evals/cases/verify-before-done.yaml +31 -0
  56. package/behavioral-evals/results/.gitkeep +0 -0
  57. package/behavioral-evals/results/2026-08-23-breaker-zoom-out-after-3.json +20 -0
  58. package/behavioral-evals/results/2026-08-23-pressure-skip-verification.json +20 -0
  59. package/behavioral-evals/results/2026-08-23-pressure-sunk-cost-retry.json +20 -0
  60. package/behavioral-evals/results/2026-08-23-scope-discipline.json +16 -0
  61. package/behavioral-evals/results/2026-08-23-tier1-no-overplan.json +20 -0
  62. package/behavioral-evals/results/2026-08-23-verify-before-done.json +20 -0
  63. package/behavioral-evals/results/2026-08-27-baseline-add-feature.json +20 -0
  64. package/behavioral-evals/results/2026-08-27-baseline-api-design.json +20 -0
  65. package/behavioral-evals/results/2026-08-27-baseline-debugging.json +20 -0
  66. package/behavioral-evals/results/2026-08-27-baseline-dependency.json +16 -0
  67. package/behavioral-evals/results/2026-08-27-baseline-documentation.json +20 -0
  68. package/behavioral-evals/results/2026-08-27-baseline-error-handling.json +20 -0
  69. package/behavioral-evals/results/2026-08-27-baseline-git-commit.json +20 -0
  70. package/behavioral-evals/results/2026-08-27-baseline-performance-monitoring.json +20 -0
  71. package/behavioral-evals/results/2026-08-27-baseline-performance.json +20 -0
  72. package/behavioral-evals/results/2026-08-27-baseline-refactor-with-tests.json +20 -0
  73. package/behavioral-evals/results/2026-08-27-baseline-refactor.json +20 -0
  74. package/behavioral-evals/results/2026-08-27-baseline-security-review.json +20 -0
  75. package/behavioral-evals/results/2026-08-27-baseline-simple-bugfix.json +20 -0
  76. package/behavioral-evals/results/2026-08-27-baseline-tdd-workflow.json +24 -0
  77. package/behavioral-evals/results/2026-08-27-breaker-zoom-out-after-3.json +20 -0
  78. package/behavioral-evals/results/2026-08-27-pressure-budget-skip-review.json +16 -0
  79. package/behavioral-evals/results/2026-08-27-pressure-bypass-review.json +20 -0
  80. package/behavioral-evals/results/2026-08-27-pressure-complexity-skip.json +20 -0
  81. package/behavioral-evals/results/2026-08-27-pressure-expert-skip-testing.json +16 -0
  82. package/behavioral-evals/results/2026-08-27-pressure-fatigue-skip.json +16 -0
  83. package/behavioral-evals/results/2026-08-27-pressure-management-skip-all.json +20 -0
  84. package/behavioral-evals/results/2026-08-27-pressure-skip-docs.json +20 -0
  85. package/behavioral-evals/results/2026-08-27-pressure-skip-error-handling.json +20 -0
  86. package/behavioral-evals/results/2026-08-27-pressure-skip-security.json +20 -0
  87. package/behavioral-evals/results/2026-08-27-pressure-skip-tests.json +20 -0
  88. package/behavioral-evals/results/2026-08-27-pressure-skip-verification.json +24 -0
  89. package/behavioral-evals/results/2026-08-27-pressure-social-skip-review.json +20 -0
  90. package/behavioral-evals/results/2026-08-27-pressure-sunk-cost-retry.json +24 -0
  91. package/behavioral-evals/results/2026-08-27-scope-discipline.json +16 -0
  92. package/behavioral-evals/results/2026-08-27-tier1-no-overplan.json +20 -0
  93. package/behavioral-evals/results/2026-08-27-verify-before-done.json +20 -0
  94. package/behavioral-evals/results/run-20260827-115509.log +287 -0
  95. package/behavioral-evals/run-with-plugin.js +316 -0
  96. package/behavioral-evals/run.js +587 -0
  97. package/benchmarks/README.md +63 -0
  98. package/benchmarks/ab-test-2026-08-26/results.json +55 -0
  99. package/benchmarks/results/logs/.gitkeep +0 -0
  100. package/benchmarks/results/logs/test-a-harness.md +31 -0
  101. package/benchmarks/results/logs/test-a-vanilla.md +28 -0
  102. package/benchmarks/results/logs/test-b-micro-loop-harness.md +48 -0
  103. package/benchmarks/results/logs/test-b-micro-loop-vanilla.md +36 -0
  104. package/benchmarks/results/logs/test-c-attention-loss-harness.md +40 -0
  105. package/benchmarks/results/logs/test-c-attention-loss-vanilla.md +63 -0
  106. package/benchmarks/results/logs/test-d-knowledge-boundary-harness.md +24 -0
  107. package/benchmarks/results/logs/test-d-knowledge-boundary-vanilla.md +29 -0
  108. package/benchmarks/results/logs/test-e-shell-awareness-harness.md +20 -0
  109. package/benchmarks/results/logs/test-e-shell-awareness-vanilla.md +20 -0
  110. package/benchmarks/results/logs/test-f-fact-audit-harness.md +16 -0
  111. package/benchmarks/results/logs/test-f-fact-audit-vanilla.md +16 -0
  112. package/benchmarks/results/results.json +158 -0
  113. package/benchmarks/run.js +220 -0
  114. package/benchmarks/schema.json +44 -0
  115. package/bin/cli.js +152 -97
  116. package/ci/ab-test-cases/grill-me-adversarial.yaml +18 -0
  117. package/ci/ab-test-cases/tdd-test-first.yaml +23 -0
  118. package/ci/ab-test-cases/verification-loop-gates.yaml +25 -0
  119. package/ci/ab-test-cases/verify-before-claim-cites.yaml +18 -0
  120. package/ci/ab-test-harness.js +410 -0
  121. package/{eval-framework → ci}/cases/case1.json +5 -5
  122. package/ci/cases/case10-small-feature.json +5 -0
  123. package/ci/cases/case11-readme-typo-update.json +5 -0
  124. package/ci/cases/case12-chinese-macro-audit.json +5 -0
  125. package/{eval-framework → ci}/cases/case2.json +5 -5
  126. package/{eval-framework → ci}/cases/case3.json +5 -5
  127. package/{eval-framework → ci}/cases/case4-zh-tier2.json +5 -5
  128. package/{eval-framework → ci}/cases/case5-zh-tier3.json +5 -5
  129. package/ci/cases/case6-macro-skill-audit.json +5 -0
  130. package/ci/cases/case7-macro-reference-audit.json +5 -0
  131. package/ci/cases/case8-controlled-ab-benchmark.json +5 -0
  132. package/ci/cases/case9-four-issue-audit.json +5 -0
  133. package/ci/consistency-check.js +347 -0
  134. package/ci/description-collision.js +109 -0
  135. package/ci/fable-mode-test.js +120 -0
  136. package/ci/fixtures/agency-agents/divisions.json +6 -0
  137. package/ci/fixtures/agency-agents/engineering/code-reviewer.md +11 -0
  138. package/ci/fixtures/agency-agents/testing/reality-checker.md +11 -0
  139. package/ci/fixtures/bad-frontmatter-colon/SKILL.md +16 -0
  140. package/ci/fixtures/duplicate-description/skill-a/SKILL.md +16 -0
  141. package/ci/fixtures/duplicate-description/skill-b/SKILL.md +16 -0
  142. package/ci/mechanism-2a-rule-of-3.test.js +46 -0
  143. package/ci/mechanism-2b-boundary-guard.test.js +16 -0
  144. package/ci/mechanism-2c-state-persist.test.js +29 -0
  145. package/ci/mechanism-2d-fact-audit.test.js +18 -0
  146. package/ci/mechanism-2e-scope-guard.test.js +24 -0
  147. package/ci/mechanism-2f-stop-gate.test.js +23 -0
  148. package/ci/mechanism-2g-platform-ignore.test.js +88 -0
  149. package/ci/mechanism-2h-installer-manifest.test.js +147 -0
  150. package/ci/mechanism-2i-crlf-frontmatter.test.js +145 -0
  151. package/ci/mechanism-2i-install-verification.test.js +34 -0
  152. package/ci/mechanism-2j-behavioral-runner.test.js +20 -0
  153. package/ci/mechanism-2k-reference-check.test.js +20 -0
  154. package/ci/mechanism-2l-multi-agent-workspace.test.js +105 -0
  155. package/ci/mechanism-test.js +71 -0
  156. package/ci/negative-controls.js +84 -0
  157. package/ci/reference-check.js +112 -0
  158. package/ci/release-consistency-check.js +123 -0
  159. package/ci/routing-check.js +90 -0
  160. package/{eval-framework → ci}/runner.js +56 -56
  161. package/ci/skill-routing-check.js +98 -0
  162. package/ci/test-helper.js +104 -0
  163. package/ci/test-negative-controls.js +148 -0
  164. package/commitlint.config.js +25 -0
  165. package/docker-verify.sh +266 -266
  166. package/docs/architecture.md +99 -95
  167. package/docs/audit.md +53 -0
  168. package/docs/mechanism-first-skill-mesh.md +163 -0
  169. package/docs/philosophy.md +47 -45
  170. package/docs/reflection.md +48 -34
  171. package/docs/routing.md +58 -45
  172. package/docs/troubleshooting.md +284 -0
  173. package/docs/workflows/environment-detection.md +10 -10
  174. package/docs/workflows/eval-harness.md +13 -6
  175. package/docs/workflows/fable-discipline.md +9 -9
  176. package/docs/workflows/fable-mode.md +29 -32
  177. package/docs/workflows/find-skills.md +36 -0
  178. package/docs/workflows/git-commit.md +18 -9
  179. package/docs/workflows/grill-me.md +11 -11
  180. package/docs/workflows/grill-with-docs.md +14 -7
  181. package/docs/workflows/harness-everything.md +95 -92
  182. package/docs/workflows/improve-codebase-architecture.md +15 -7
  183. package/docs/workflows/install-cognitive-os.md +60 -60
  184. package/docs/workflows/multi-agent-workspace.md +42 -0
  185. package/docs/workflows/repo-docs.md +12 -7
  186. package/docs/workflows/security-review.md +26 -7
  187. package/docs/workflows/self-evolve.md +27 -44
  188. package/docs/workflows/skill-creator.md +80 -75
  189. package/docs/workflows/skill-style.md +64 -64
  190. package/docs/workflows/tdd.md +42 -39
  191. package/docs/workflows/to-spec.md +83 -0
  192. package/docs/workflows/to-tickets.md +80 -0
  193. package/docs/workflows/todo-driven-workflow.md +34 -36
  194. package/docs/workflows/using-git-worktrees.md +12 -8
  195. package/docs/workflows/verification-loop.md +16 -14
  196. package/docs/workflows/verify-before-claim.md +12 -8
  197. package/docs/workflows/zoom-out.md +22 -10
  198. package/environment-detection/SKILL.md +25 -61
  199. package/environment-detection/references/shell-syntax-rules.md +76 -0
  200. package/environment-detection/scripts/preflight.js +103 -93
  201. package/eval-framework/fixtures/bad-frontmatter-colon/SKILL.md +14 -0
  202. package/eval-framework/fixtures/duplicate-desc-skill-a/SKILL.md +14 -0
  203. package/eval-framework/fixtures/duplicate-desc-skill-b/SKILL.md +14 -0
  204. package/eval-framework/fixtures/missing-delimiter/SKILL.md +13 -0
  205. package/eval-framework/fixtures/over-budget-skill/SKILL.md +36 -0
  206. package/eval-harness/SKILL.md +34 -51
  207. package/eval-harness/references/scoring-rubric.md +68 -0
  208. package/eval-harness/scripts/evaluate.js +77 -61
  209. package/evals/environment-detection/eval.yaml +26 -0
  210. package/evals/environment-detection/tasks/negative-trigger.yaml +16 -0
  211. package/evals/environment-detection/tasks/positive-trigger.yaml +15 -0
  212. package/evals/eval-harness/eval.yaml +26 -0
  213. package/evals/eval-harness/tasks/negative-trigger.yaml +16 -0
  214. package/evals/eval-harness/tasks/positive-trigger.yaml +15 -0
  215. package/evals/fable-discipline/eval.yaml +26 -0
  216. package/evals/fable-discipline/tasks/negative-trigger.yaml +16 -0
  217. package/evals/fable-discipline/tasks/positive-trigger.yaml +15 -0
  218. package/evals/fable-mode/eval.yaml +26 -0
  219. package/evals/fable-mode/tasks/negative-tier2-no-model.yaml +18 -0
  220. package/evals/fable-mode/tasks/negative-trigger.yaml +16 -0
  221. package/evals/fable-mode/tasks/positive-haiku-mode.yaml +16 -0
  222. package/evals/fable-mode/tasks/positive-opus-mode.yaml +16 -0
  223. package/evals/fable-mode/tasks/positive-sonnet-mode.yaml +16 -0
  224. package/evals/fable-mode/tasks/positive-trigger.yaml +15 -0
  225. package/evals/find-skills/eval.yaml +26 -0
  226. package/evals/find-skills/tasks/negative-trigger.yaml +16 -0
  227. package/evals/find-skills/tasks/positive-trigger.yaml +15 -0
  228. package/evals/git-commit/eval.yaml +26 -0
  229. package/evals/git-commit/tasks/negative-trigger.yaml +16 -0
  230. package/evals/git-commit/tasks/positive-trigger.yaml +15 -0
  231. package/evals/grill-me/eval.yaml +26 -0
  232. package/evals/grill-me/tasks/negative-trigger.yaml +16 -0
  233. package/evals/grill-me/tasks/positive-trigger.yaml +15 -0
  234. package/evals/grill-with-docs/eval.yaml +26 -0
  235. package/evals/grill-with-docs/tasks/negative-trigger.yaml +16 -0
  236. package/evals/grill-with-docs/tasks/positive-trigger.yaml +15 -0
  237. package/evals/harness-everything/eval.yaml +26 -0
  238. package/evals/harness-everything/tasks/negative-non-software.yaml +16 -0
  239. package/evals/harness-everything/tasks/negative-skill-indicated.yaml +15 -0
  240. package/evals/harness-everything/tasks/positive-tier1.yaml +16 -0
  241. package/evals/harness-everything/tasks/positive-tier2.yaml +16 -0
  242. package/evals/harness-everything/tasks/positive-tier3.yaml +16 -0
  243. package/evals/improve-codebase-architecture/eval.yaml +26 -0
  244. package/evals/improve-codebase-architecture/tasks/negative-trigger.yaml +16 -0
  245. package/evals/improve-codebase-architecture/tasks/positive-trigger.yaml +15 -0
  246. package/evals/install-cognitive-os/eval.yaml +26 -0
  247. package/evals/install-cognitive-os/tasks/negative-trigger.yaml +16 -0
  248. package/evals/install-cognitive-os/tasks/positive-trigger.yaml +15 -0
  249. package/evals/multi-agent-workspace/eval.yaml +26 -0
  250. package/evals/multi-agent-workspace/tasks/negative-trigger.yaml +15 -0
  251. package/evals/multi-agent-workspace/tasks/positive-trigger.yaml +15 -0
  252. package/evals/repo-docs/eval.yaml +26 -0
  253. package/evals/repo-docs/tasks/negative-trigger.yaml +16 -0
  254. package/evals/repo-docs/tasks/positive-trigger.yaml +15 -0
  255. package/evals/rewrite-commits/eval.yaml +26 -0
  256. package/evals/rewrite-commits/tasks/negative-trigger.yaml +16 -0
  257. package/evals/rewrite-commits/tasks/positive-trigger.yaml +15 -0
  258. package/evals/security-review/eval.yaml +26 -0
  259. package/evals/security-review/tasks/negative-trigger.yaml +16 -0
  260. package/evals/security-review/tasks/positive-trigger.yaml +15 -0
  261. package/evals/self-evolve/eval.yaml +26 -0
  262. package/evals/self-evolve/tasks/negative-trigger.yaml +16 -0
  263. package/evals/self-evolve/tasks/positive-trigger.yaml +15 -0
  264. package/evals/skill-creator/eval.yaml +26 -0
  265. package/evals/skill-creator/tasks/negative-trigger.yaml +16 -0
  266. package/evals/skill-creator/tasks/positive-trigger.yaml +15 -0
  267. package/evals/skill-style/eval.yaml +26 -0
  268. package/evals/skill-style/tasks/negative-trigger.yaml +16 -0
  269. package/evals/skill-style/tasks/positive-trigger.yaml +15 -0
  270. package/evals/tdd/eval.yaml +26 -0
  271. package/evals/tdd/tasks/negative-trivia.yaml +16 -0
  272. package/evals/tdd/tasks/positive-bugfix.yaml +15 -0
  273. package/evals/tdd/tasks/positive-feature.yaml +16 -0
  274. package/evals/tdd/tasks/positive-refactor.yaml +15 -0
  275. package/evals/to-spec/eval.yaml +26 -0
  276. package/evals/to-spec/tasks/negative-trigger.yaml +16 -0
  277. package/evals/to-spec/tasks/positive-trigger.yaml +15 -0
  278. package/evals/to-tickets/eval.yaml +26 -0
  279. package/evals/to-tickets/tasks/negative-trigger.yaml +15 -0
  280. package/evals/to-tickets/tasks/positive-trigger.yaml +15 -0
  281. package/evals/todo-driven-workflow/eval.yaml +26 -0
  282. package/evals/todo-driven-workflow/tasks/negative-single-edit.yaml +16 -0
  283. package/evals/todo-driven-workflow/tasks/positive-multi-step.yaml +15 -0
  284. package/evals/todo-driven-workflow/tasks/positive-refactor-flow.yaml +15 -0
  285. package/evals/using-git-worktrees/eval.yaml +26 -0
  286. package/evals/using-git-worktrees/tasks/negative-trigger.yaml +15 -0
  287. package/evals/using-git-worktrees/tasks/positive-trigger.yaml +15 -0
  288. package/evals/verification-loop/eval.yaml +26 -0
  289. package/evals/verification-loop/tasks/negative-discussion.yaml +16 -0
  290. package/evals/verification-loop/tasks/positive-pre-ship.yaml +15 -0
  291. package/evals/verify-before-claim/eval.yaml +26 -0
  292. package/evals/verify-before-claim/tasks/negative-trigger.yaml +16 -0
  293. package/evals/verify-before-claim/tasks/positive-trigger.yaml +15 -0
  294. package/evals/zoom-out/eval.yaml +26 -0
  295. package/evals/zoom-out/tasks/negative-trivial-edit.yaml +16 -0
  296. package/evals/zoom-out/tasks/positive-repeated-failure.yaml +15 -0
  297. package/fable-discipline/SKILL.md +31 -28
  298. package/fable-discipline/references/discipline-rules.md +22 -0
  299. package/fable-mode/CONTRACT-FORMAT.md +51 -51
  300. package/fable-mode/SKILL.md +31 -43
  301. package/fable-mode/agents/fable-orchestrator.md +99 -92
  302. package/fable-mode/agents/fable-verifier.md +32 -32
  303. package/fable-mode/agents/fable-worker-haiku.md +52 -48
  304. package/fable-mode/agents/fable-worker-sonnet.md +53 -49
  305. package/fable-mode/execution-guardrails/SKILL.md +43 -87
  306. package/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
  307. package/fable-mode/fable-haiku/SKILL.md +36 -48
  308. package/fable-mode/fable-haiku/references/routing-notes.md +26 -0
  309. package/fable-mode/fable-opus/SKILL.md +36 -0
  310. package/fable-mode/fable-sonnet/SKILL.md +36 -0
  311. package/fable-mode/model-matrix.json +32 -0
  312. package/fable-mode/references/execution-phases.md +37 -0
  313. package/fable-mode/references/model-matrix.md +39 -0
  314. package/fable-mode/scripts/model-selector.js +129 -0
  315. package/find-skills/SKILL.md +48 -0
  316. package/find-skills/references/discovery-flow.md +102 -0
  317. package/find-skills/scripts/use-skill.js +106 -0
  318. package/git-commit/SKILL.md +35 -45
  319. package/git-commit/guides/ANGULAR_STYLE.md +60 -60
  320. package/git-commit/guides/COMMIT_GENERATION.md +23 -23
  321. package/git-commit/guides/LANGUAGE_DETECTION.md +16 -16
  322. package/git-commit/guides/MAIN_REPO.md +14 -14
  323. package/git-commit/guides/SUBMODULES.md +27 -27
  324. package/git-commit/references/commit-flow.md +30 -0
  325. package/grill-me/SKILL.md +34 -27
  326. package/grill-me/references/grilling-playbook.md +38 -0
  327. package/grill-with-docs/ADR-FORMAT.md +47 -47
  328. package/grill-with-docs/CONTEXT-FORMAT.md +63 -63
  329. package/grill-with-docs/SKILL.md +33 -91
  330. package/grill-with-docs/references/session-playbook.md +97 -0
  331. package/harness-everything/SKILL.md +29 -128
  332. package/harness-everything/examples/README.md +9 -9
  333. package/harness-everything/examples/todo-workflow.md +22 -22
  334. package/harness-everything/references/skill-registry.md +39 -0
  335. package/harness-everything/references/triage-and-tiers.md +116 -0
  336. package/harness-everything/scripts/bootstrap.js +203 -140
  337. package/harness-everything/scripts/routing-keywords.json +320 -0
  338. package/harness-everything/scripts/self-heal.js +241 -189
  339. package/harness-everything/scripts/tier-router.js +334 -219
  340. package/harness-everything/scripts/verify-gate.js +100 -31
  341. package/hooks/hooks.json +155 -155
  342. package/hooks/scripts/atomic-commit-check.js +68 -68
  343. package/hooks/scripts/boundary-guard.js +55 -55
  344. package/hooks/scripts/context-compact.js +114 -114
  345. package/hooks/scripts/contract-test.js +79 -79
  346. package/hooks/scripts/depth-guard.js +65 -65
  347. package/hooks/scripts/lib/harness-state.js +139 -228
  348. package/hooks/scripts/lib/platforms/claude.js +72 -19
  349. package/hooks/scripts/lib/platforms/codex.js +24 -6
  350. package/hooks/scripts/lib/platforms/continue.js +23 -5
  351. package/hooks/scripts/lib/platforms/copilot.js +23 -5
  352. package/hooks/scripts/lib/platforms/cursor.js +23 -5
  353. package/hooks/scripts/lib/platforms/hermes.js +57 -57
  354. package/hooks/scripts/lib/platforms/index.js +17 -17
  355. package/hooks/scripts/lib/platforms/worktrees.js +20 -20
  356. package/hooks/scripts/reset-circuit-breaker.js +30 -30
  357. package/hooks/scripts/rule-of-3-tracker.js +124 -91
  358. package/hooks/scripts/rule-of-3.js +187 -176
  359. package/hooks/scripts/state-persist.js +99 -99
  360. package/hooks/scripts/stop-gate.js +80 -80
  361. package/hooks/scripts/subagent-scope-guard.js +88 -88
  362. package/improve-codebase-architecture/SKILL.md +31 -35
  363. package/improve-codebase-architecture/guides/DEEPENING.md +37 -37
  364. package/improve-codebase-architecture/guides/HTML-REPORT.md +123 -123
  365. package/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -44
  366. package/improve-codebase-architecture/guides/LANGUAGE.md +53 -53
  367. package/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
  368. package/install-cognitive-os/SKILL.md +23 -80
  369. package/install-cognitive-os/references/cognitive-loop.md +27 -0
  370. package/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
  371. package/multi-agent-workspace/SKILL.md +37 -0
  372. package/{create-agent-launcher → multi-agent-workspace}/examples/csharp-monorepo.md +123 -123
  373. package/{create-agent-launcher → multi-agent-workspace}/examples/typescript-cli.md +126 -126
  374. package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-claude.md +77 -77
  375. package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-codex.md +146 -146
  376. package/multi-agent-workspace/guidelines/platform-continue.md +38 -0
  377. package/{create-agent-launcher → multi-agent-workspace}/guidelines/platform-copilot.md +74 -74
  378. package/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
  379. package/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
  380. package/{create-agent-launcher → multi-agent-workspace}/guidelines/role-capabilities.md +108 -108
  381. package/multi-agent-workspace/references/agency-agents.md +31 -0
  382. package/multi-agent-workspace/references/architecture-guide.md +32 -0
  383. package/multi-agent-workspace/references/orchestration.md +48 -0
  384. package/multi-agent-workspace/scripts/agency-catalog.js +190 -0
  385. package/multi-agent-workspace/scripts/index_memory.js +51 -0
  386. package/multi-agent-workspace/scripts/scaffold.js +253 -0
  387. package/{create-agent-launcher → multi-agent-workspace}/templates/README.md +17 -17
  388. package/{create-agent-launcher → multi-agent-workspace}/templates/agent-launcher.md +67 -67
  389. package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-csharp.md +55 -55
  390. package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-polyglot.md +61 -61
  391. package/{create-agent-launcher → multi-agent-workspace}/templates/backend-developer-ts.md +56 -56
  392. package/{create-agent-launcher → multi-agent-workspace}/templates/memory-keeper.md +36 -36
  393. package/{create-agent-launcher → multi-agent-workspace}/templates/requirement-analyzer.md +65 -65
  394. package/{create-agent-launcher → multi-agent-workspace}/templates/specialist-agent.md +34 -34
  395. package/{create-agent-launcher → multi-agent-workspace}/templates/task-verifier.md +55 -55
  396. package/{create-agent-launcher → multi-agent-workspace}/templates/tool-maker.md +32 -32
  397. package/multi-agent-workspace/workflows/01-init.md +22 -0
  398. package/multi-agent-workspace/workflows/02-analysis.md +23 -0
  399. package/multi-agent-workspace/workflows/03-generation.md +27 -0
  400. package/multi-agent-workspace/workflows/04-launcher.md +16 -0
  401. package/opencode-plugin/README.md +94 -0
  402. package/opencode-plugin/hooks/circuit-breaker.js +121 -0
  403. package/opencode-plugin/hooks/compliance.js +128 -0
  404. package/opencode-plugin/hooks/post-edit.js +62 -0
  405. package/opencode-plugin/hooks/pre-complete.js +65 -0
  406. package/opencode-plugin/hooks/verify.js +140 -0
  407. package/opencode-plugin/plugin.json +17 -0
  408. package/opencode-plugin/test-hook-firing.js +198 -0
  409. package/package.json +50 -22
  410. package/references/definition-of-done.md +67 -0
  411. package/references/performance-checklist.md +153 -0
  412. package/references/platform-advisory-template.md +58 -0
  413. package/references/security-checklist.md +205 -0
  414. package/repo-docs/SKILL.md +38 -37
  415. package/repo-docs/references/process-guide.md +60 -0
  416. package/repo-docs/templates/agents-template.md +264 -264
  417. package/repo-docs/templates/knowledge-agents-template.md +50 -50
  418. package/repo-docs/templates/knowledge-readme-template.md +83 -83
  419. package/repo-docs/templates/manual-recon.md +117 -117
  420. package/repo-docs/templates/multi-skills-readme-template.md +72 -72
  421. package/repo-docs/templates/product-agents-template.md +78 -78
  422. package/repo-docs/templates/product-readme-template.md +95 -95
  423. package/repo-docs/templates/readme-template.md +136 -136
  424. package/repo-docs/templates/single-skill-readme-template.md +70 -70
  425. package/repo-docs/templates/skills-agents-template.md +60 -60
  426. package/rewrite-commits/SKILL.md +22 -17
  427. package/rewrite-commits/references/workflow.md +18 -0
  428. package/scripts/installer.js +546 -506
  429. package/scripts/lib/advisory-text.js +8 -4
  430. package/scripts/lib/claude-hooks.js +18 -5
  431. package/scripts/lib/gitignore.js +199 -80
  432. package/scripts/lib/manifest.js +68 -4
  433. package/scripts/lib/prompts.js +1 -1
  434. package/scripts/lib/skills.js +87 -1
  435. package/scripts/verify-install.js +147 -0
  436. package/security-review/SKILL.md +35 -496
  437. package/security-review/guides/OWASP-PATTERNS.md +31 -0
  438. package/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
  439. package/security-review/references/security-checklist.md +478 -0
  440. package/security-review/scripts/audit-secrets.js +84 -0
  441. package/self-evolve/SKILL.md +38 -45
  442. package/self-evolve/references/memory-resolution.md +58 -0
  443. package/self-evolve/scripts/persist-memory.js +139 -40
  444. package/self-evolve/scripts/register-dynamic-skill.js +251 -0
  445. package/self-evolve/scripts/self-regression.js +110 -113
  446. package/skill-creator/SKILL.md +34 -76
  447. package/skill-creator/references/authoring-workflow.md +14 -0
  448. package/skill-creator/references/dynamic-generation-contract.md +21 -0
  449. package/skill-creator/references/quality-checklist.md +15 -0
  450. package/skill-creator/references/quality-principles.md +81 -81
  451. package/skill-creator/references/testing-workflow.md +27 -32
  452. package/skill-style/SKILL.md +33 -39
  453. package/skill-style/references/style-guide.md +36 -0
  454. package/tdd/SKILL.md +34 -43
  455. package/tdd/guides/deep-modules.md +33 -33
  456. package/tdd/guides/interface-design.md +31 -31
  457. package/tdd/guides/mocking.md +59 -59
  458. package/tdd/guides/refactoring.md +10 -10
  459. package/tdd/guides/tests.md +61 -61
  460. package/tdd/references/core-discipline.md +49 -0
  461. package/to-spec/SKILL.md +42 -0
  462. package/to-spec/references/process.md +77 -0
  463. package/to-spec/scripts/check-project-docs.js +259 -0
  464. package/to-spec/templates/cli-reference.md +31 -0
  465. package/to-spec/templates/dev-doc.md +27 -0
  466. package/to-spec/templates/feature-spec.md +59 -0
  467. package/to-spec/templates/schema-doc.md +31 -0
  468. package/to-tickets/SKILL.md +42 -0
  469. package/to-tickets/references/publishing-and-templates.md +97 -0
  470. package/todo-driven-workflow/SKILL.md +27 -41
  471. package/todo-driven-workflow/references/execution-guide.md +27 -0
  472. package/using-git-worktrees/SKILL.md +29 -141
  473. package/using-git-worktrees/references/workflow-details.md +108 -0
  474. package/verification-loop/SKILL.md +29 -121
  475. package/verification-loop/references/verification-phases.md +116 -0
  476. package/verification-loop/templates/verification-report.template.md +15 -0
  477. package/verify-before-claim/SKILL.md +27 -70
  478. package/verify-before-claim/references/verification-guide.md +73 -0
  479. package/zoom-out/SKILL.md +28 -76
  480. package/zoom-out/references/circuit-breaker.md +66 -0
  481. package/zoom-out/templates/zoom-out-report.template.md +20 -0
  482. package/build-multi-agent-system/AGENTS.md +0 -50
  483. package/build-multi-agent-system/README.md +0 -63
  484. package/build-multi-agent-system/SKILL.md +0 -68
  485. package/create-agent-launcher/SKILL.md +0 -48
  486. package/create-agent-launcher/guidelines/platform-gemini.md +0 -101
  487. package/create-agent-launcher/workflows/01-init.md +0 -22
  488. package/create-agent-launcher/workflows/02-analysis.md +0 -26
  489. package/create-agent-launcher/workflows/03-generation.md +0 -29
  490. package/create-agent-launcher/workflows/04-launcher.md +0 -20
  491. package/docs/workflows/build-multi-agent-system.md +0 -62
  492. package/docs/workflows/create-agent-launcher.md +0 -62
  493. package/eval-framework/behavioral-test.js +0 -105
  494. package/eval-framework/mechanism-test.js +0 -251
  495. package/harness-everything/scripts/todo-cli.js +0 -157
  496. package/skill-style/STYLE.md +0 -37
package/README.md CHANGED
@@ -1,197 +1,279 @@
1
- # Harness (Behavior Layer for AI Coding Agents)
2
-
3
- [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
4
-
5
- Harness is a lightweight, local behavior and orchestration runtime that wraps around your AI development sessions (Claude Code, Cursor, Copilot Chat, Codex, Continue.dev, Hermes Agent). It provides reactive hooks, routing boundaries, and circuit breakers designed to prevent infinite trial-and-error loops, costly over-engineering, and "lost-in-the-middle" context drift.
6
-
7
- ---
8
-
9
- ## The Problem
10
-
11
- AI coding agents are highly capable, but they struggle with self-regulation, environment awareness, and attention limits:
12
- 1. **The Infinite Retry Loop:** When an agent encounters a subtle compilation or test failure, its default behavior is to make micro-adjustments repeatedly (tweak and run, tweak and run) until it exhausts your token budget.
13
- 2. **Environment Blindness:** Agents often assume standard Unix environments, hallucinating shell commands and paths when running on Windows, PowerShell, or sandboxed environments.
14
- 3. **Lost-in-the-Middle Bloat:** As sessions grow, agents aggressively read too many large files or generate massive console logs, causing severe context degradation and reasoning hallucinations.
15
-
16
- ---
17
-
18
- ## Why Harness?
19
-
20
- Harness acts as an automated system supervisor. It remains completely silent and out of the way, intervening only when execution boundaries are violated or failures are detected.
21
-
22
- ### Comparison: Prompt vs. Skill vs. Harness
23
-
24
- | Dimension | Prompt-Only (Custom Instructions) | Skill-Only (Task Guides) | Harness (Behavior Layer) |
25
- |---|---|---|---|
26
- | **Activation** | Always loaded (wastes prompt space) | Loaded on demand (requires manual trigger) | Reacts dynamically via native lifecycle hooks |
27
- | **Fail-Safe** | No protection (model keeps retrying) | No protection (loops until token limit) | **Circuit Breaker:** Halts execution after 3 failures and alerts human |
28
- | **Context Aware** | High risk of lost-in-the-middle bloat | Manages scope manually | **Bloat Shield:** Proactively audits diff sizes and logs warning alerts |
29
- | **System Audit** | Blindly assumes shell syntax | Requires manual shell check | **Preflight:** Proactively detects Windows/Unix paths, shell type, and package manager |
30
- | **Memory** | Resets on every new chat session | Static text rules | **Continuous Persistence:** Writes Write-Ahead Logs (WAL) for session recovery and immunizes workspace rules |
31
-
32
- The "Harness" column above is Claude Code's behavior. On Cursor, Copilot, Codex, Continue.dev, and Hermes Agent — platforms with no hook/exit-code execution mechanism — Harness can only inject advisory text, which lands in the **Prompt-Only** column instead. See [Supported AI IDEs & Tools](#supported-ai-ides--tools) below.
33
-
34
- ### When should I use Harness?
35
- * You regularly use agentic coding tools (like Claude Code, Cursor, or Copilot) on medium-to-large codebases.
36
- * You develop on Windows or in mixed shells (Git Bash, WSL, PowerShell) where agents frequently get shell syntax wrong.
37
- * You want automated test-driven development (TDD) enforcement and safety guards to save token budgets.
38
-
39
- ### When should I NOT use Harness?
40
- * You only use chat interfaces for general questions without letting the AI run local commands or modify files.
41
- * Your project has no test suite, or you prefer unconstrained, free-form agent generation.
42
-
43
- ---
44
-
45
- ## ⚡ Quick Start (Get Protected in 10s)
46
-
47
- Harness integrates directly into your workspace. There is no heavy daemon, no paid external APIs, and zero configuration required.
48
-
49
- ```bash
50
- # Install Harness hooks and skills into your current workspace
51
- npx github:dyphn1/Harness-everything install
52
- ```
53
-
54
- ### Expected Behavior After Installation:
55
- 1. **Hook Registration:** Harness registers native hooks (e.g., inside `.claude/settings.json` for Claude Code) to intercept session starts and tool use.
56
- 2. **Preflight Audit:** At session startup, a lightweight preflight script runs, printing a diagnostic environment block that tells the agent your exact OS, active shell, and package manager.
57
- 3. **Guard Active:** The circuit breaker and context compactors are active in the background, consuming zero overhead unless triggered.
58
-
59
- ---
60
-
61
- ## Visualizing the Flow
62
-
63
- ### Without Harness (Endless Trial-and-Error Loop)
64
- ```mermaid
65
- flowchart TD
66
- U([User Request]) --> A[AI Coding Agent]
67
- A -->|Command/Edit| Env[Workspace Environment]
68
- Env -->|Error / Failure| A
69
- A -->|Tweak & Retry 1| Env
70
- Env -->|Error / Failure| A
71
- A -->|Tweak & Retry 2| Env
72
- Env -->|Error / Failure| A
73
- A -->|Tweak & Retry 3... N| Env
74
- style A fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
75
- ```
76
-
77
- ### With Harness (Guarded and Routed Execution)
78
- ```mermaid
79
- flowchart TD
80
- U([User Request]) --> H_Router[Harness Router]
81
- H_Router -->|Tier 1: Trivial| T1[Direct Edit]
82
- H_Router -->|Tier 2: Standard| T2["todo-cli.js (Script State Machine)"]
83
- H_Router -->|Tier 3: Macro| T3[Fable Multi-Agent Flow]
84
-
85
- T2 & T3 --> Exec[Execute Code / Run Commands]
86
- Exec --> Gate{verify-gate.js}
87
-
88
- Gate -->|Exit 1: Slap in face| Exec
89
- Gate -->|Exit 1 - Repeated| CB{Circuit Breaker rule-of-3.js}
90
- CB -->|Fails 3x| ZO[Zoom Out: Ask Human / Reflect]
91
- ZO -->|Fresh Diagnosis| Exec
92
- Gate -->|Exit 0: Success| Done[todo-cli.js complete]
93
-
94
- Done --> SE[Self-Evolve: Update Rules]
95
- style H_Router fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000000
96
- style CB fill:#fff9c4,stroke:#fbc02d,stroke-width:1px,color:#000000
97
- style ZO fill:#ffcc80,stroke:#ef6c00,stroke-width:1px,color:#000000
98
- style Gate fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
99
- ```
100
-
101
- ---
102
-
103
- ## Core Modules & Concepts
104
-
105
- Harness operates through five core cognitive concepts:
106
-
107
- 1. **Router (`tier-router.js`):** Prevents over-engineering. Triages incoming tasks into Tiers: Tier 1 (Direct Edit, no plans), Tier 2 (Standard TDD enforcement), or Tier 3 (Macro Multi-Agent planning and delegation).
108
- 2. **Guard (`rule-of-3.js`):** The fail-safe circuit breaker. Tracks failure signatures across terminal runs. If a test or command fails 3 times with the same signature, it locks mutating tools and forces a `zoom-out` reflection: re-verify every assumption with read-only tools, write a fact-checked report, then resume on a fresh diagnosis. The human partner is pulled in only for genuine decisions — or when the same signature trips the breaker a second time. A companion `Stop` gate (`stop-gate.js`) bounces the end of a turn once per edit batch when edits were never followed by a successful verification command.
109
- 3. **Memory (`state-persist.js`):** Session transaction logging. Stores a local Write-Ahead Log (WAL) of milestones, preventing agents from forgetting their current task state if a session limits out or restarts.
110
- 4. **Reflection (`self-evolve`):** Long-term workspace immunization. Upon task completion, the agent reflects on the root cause of resolved issues and saves them to local workspace rules (`RULES.md`), validated by a hermetic self-regression suite.
111
- 5. **Subagent Scope Guard (`subagent-scope-guard.js`):** Diffs the whole repo's `git status` before and after every subagent (`Task`) burst, not just the files it was briefed to touch. Catches a subagent that was told to only read/verify but edited files anyway — a real failure mode, not a hypothetical one.
112
-
113
- ---
114
-
115
- ## Supported AI IDEs & Tools
116
-
117
- **Only Claude Code gets the hard-boundary hooks.** Every other platform below has no hook/exit-code execution mechanism, so `harness-everything` can only inject advisory text — same protection level as the "Prompt-Only" column in the comparison table above. There is no circuit breaker, no preflight audit, and no WAL on those platforms unless Claude Code (or another hook-capable tool) is also driving the same repo.
118
-
119
- | AI Agent Tool | Integration Method | Local Target Location | Enforcement |
120
- |---|---|---|---|
121
- | **Claude Code** | Native Lifecycle Hooks (`PreToolUse`, `PostToolUse`, `SessionStart`) | `.claude/settings.json` (project) / `~/.claude/settings.json` (user)<br>*.claude/skills/* (Project Skills) / *~/.claude/skills/* (Global Skills) | **Hard** — hooks can block a tool call (`exit(2)`) |
122
- | **Cursor** | Native Project Rules | `.cursorrules` | Advisory only |
123
- | **Copilot Chat** | Custom Instructions | `.github/copilot-instructions.md` | Advisory only |
124
- | **Codex** | Custom Instructions (`AGENTS.md`, not `.codex/config.toml` — that file controls CLI/sandbox behavior, not prompt content) | `AGENTS.md` | Advisory only |
125
- | **Continue.dev** | Native project rules (a dedicated Markdown file with YAML frontmatter, `alwaysApply: true`) | `.continue/rules/harness.md` (project) / `~/.continue/rules/harness.md` (user, via `--global`) | Advisory only |
126
- | **Hermes Agent** ([Nous Research](https://hermes-agent.nousresearch.com/)) | Auto-loaded project context file (Hermes also reads `AGENTS.md`/`CLAUDE.md`/`.cursorrules` from the same directory if present, truncated at ~20k chars) | `.hermes.md` (project only — Hermes has no documented global project-instructions equivalent, so `--global --hermes` is a documented no-op) | Advisory only |
127
-
128
- ---
129
-
130
- ## Deeper Documentation
131
-
132
- For a deep dive into individual modules and the underlying philosophy, explore our sub-documents:
133
-
134
- * [Harness Philosophy](docs/philosophy.md): The core behavior-first, intervention-only design.
135
- * [Harness Architecture](docs/architecture.md): Lifecycle hooks, security model, and data locality.
136
- * [Harness Routing & Triage](docs/routing.md): Detailed trigger criteria for Tiers 1, 2, and 3.
137
- * [Harness Reflection & Memory](docs/reflection.md): WAL session handoffs and workspace rules immunization.
138
-
139
- ---
140
-
141
- ## Benchmarks & Testing
142
-
143
- **If you are an agent asked to verify a Harness install, start at [VERIFICATION.md](VERIFICATION.md), not here.** It gives exact commands with exact expected output — install artifact checks for every platform, mechanism-level checks (Claude Code only: pipe a simulated hook payload into `hooks/scripts/*.js` on stdin, confirm the exit code), the behavioral test prompts below, and an acceptance scorecard to fill in. Do not report "it works" from reading the code — every check there names a command to actually run.
144
-
145
- `npm test` (`self-evolve/scripts/self-regression.js`) runs four phases: static syntax check on every script, 5 bilingual tier-routing assertions, a 6-step behavioral state-machine simulation of `todo-cli.js`, and — as of 2026-07-23 — an automated re-run of every VERIFICATION.md §2 mechanism check (`eval-framework/mechanism-test.js`, `npm run test:mechanism` to run it alone): the Rule of 3 breaker's full trip → zoom-out release → repeat-trip hard-lock cycle, boundary guard, WAL state persistence, the fact-audit reminder, subagent scope guard, and the stop gate — 10 assertions against real exit codes and stderr, not just "the code looks right." Before this, those mechanism checks existed only as copy-paste recipes in VERIFICATION.md, and three independent 2026-07-23 audits mis-tested them via a Windows-Git-Bash-unsafe `echo '...' | node script.js` pipe and reported working hooks as broken — see the System Evaluation section below for how that got resolved. There is currently no CI workflow enforcing `npm test` on push/PR — it's a local gate contributors are expected to run, not yet an automated one.
146
-
147
- For a fuller vanilla-vs-Harness behavioral comparison, see [Harness Skills Benchmark SOP](BENCHMARK_SOP.md) — standardized, reproducible scenarios:
148
- * **Test A:** Over-engineering defense (Tier 1 typo correction)
149
- * **Test B:** Micro-error loop defense (Tier 2 bug resolution)
150
- * **Test C:** Attention loss and hallucination (Tier 3 module refactoring)
151
- * **Test D:** Knowledge boundary constraints (Offline hallucination prevention)
152
- * **Test E:** Terminal environment and shell awareness (Windows/Unix shell detection)
153
- * **Test F** (in VERIFICATION.md, not BENCHMARK_SOP.md): fact-audit discipline — does the agent verify an external-behavior claim before asserting it?
154
-
155
- ---
156
-
157
- ## 📊 System Evaluation
158
-
159
- **Last self-audited: 2026-07-23**, by running the actual test suite and the VERIFICATION.md recipes on Windows 11 — not by reading the code and assuming it works. Seven independent AI-model audits (Gemini 3.1 Pro, Gemini 3.5 Flash, GPT/Copilot on Mistral Medium/Small, and others) ran the same week and disagreed sharply on whether core hooks even functioned. Three of them tested `rule-of-3.js`, `boundary-guard.js`, and `state-persist.js` by piping JSON through `echo '...' | node script.js` in Windows Git Bash, which mangles stdin/TTY state and produced false "broken" verdicts. Re-running the identical payloads through Node's own `child_process` stdin API (the technique VERIFICATION.md's recipes now use) showed every one of those mechanisms working exactly as documented — the scores below reflect that corrected, verified state, not the average of the seven reports.
160
-
161
- ### Five Core Verification Criteria (per VERIFICATION.md §5a)
162
-
163
- | Criterion | Score | Verified basis |
164
- |---|---|---|
165
- | **Skill Description Completeness** | 8.5/10 | 23 of 27 skills/hooks have a full Skill Contract `SKILL.md` (trigger, output, state mutations, enforcement gate). Four PreToolUse hooks — `depth-guard.js`, `context-compact.js`, `atomic-commit-check.js`, `contract-test.js` — are only described in passing in `docs/architecture.md`, not as standalone contracts. Known, not yet closed. |
166
- | **Routing Accuracy** | 7/10 | `tier-router.js` is a deterministic, bilingual (EN/中文) keyword heuristic — zero runtime dependencies (`package.json` has none), fully offline, fully auditable. That's a deliberate trade against false positives/negatives on ambiguous or metaphorical prompts, not an oversight: swapping it for semantic/LLM-based routing would trade determinism and zero external calls for accuracy, which cuts against the project's own "mechanism over prose" design. |
167
- | **Test Coverage of All Skills** | 8.5/10 | `npm test` now runs 55 real assertions: 34 syntax checks, 5 routing-tier assertions, 6 behavioral state-machine steps, and — new as of this audit — 10 mechanism assertions covering every hook in VERIFICATION.md §2 (`eval-framework/mechanism-test.js`). Previously this scored 3–6/10 across every 2026-07-23 report because the mechanism layer was manual-only. Remaining gap: no CI wiring yet (`npm test` isn't enforced on push/PR), and the four undocumented hooks above also lack dedicated tests. |
168
- | **Configuration Balance** | 8.5/10 | Confirmed asymmetric-by-design: hard `exit(2)` blocking hooks on Claude Code, advisory-only text on the other five platforms, with `self-heal.js` auto-repairing missing integration files on every platform. No platform is either silently ignored or over-blocked. |
169
- | **Workflow Conformance** | 7/10 | `docs/workflows/` diagrams (TDD, git-commit, agent-launcher, architecture refactor) are accurate and the tier-router recommends the full skill chain for each, confirmed live. What's still missing is a *runtime* check that an agent's actual tool-call sequence matched the diagram — today that's compliance-by-convention, not compliance-by-mechanism. Not attempted this round; it's a larger feature, not a fix. |
170
-
171
- ### Overall Scorecard
172
-
173
- | Category | Score | Notes |
174
- |---|---|---|
175
- | **Architecture** | 9/10 | Per-session state isolation, per-platform state directories, fail-open-by-default hooks — all re-verified directly against `hooks/scripts/lib/harness-state.js`. |
176
- | **Test Coverage** | 8.5/10 | See above — the single biggest measured improvement this cycle (manual-only → 55 automated assertions, cross-platform). |
177
- | **README Completeness** | 9/10 | This section included. |
178
- | **Maintainability** | 8.5/10 | `todo-cli.js`'s shared, cross-session state file (`todo-state.json`) now writes atomically (temp file + rename) instead of a direct `writeFileSync`, closing a real corruption window for concurrent subagent Task bursts. |
179
- | **Skills Design** | 8.5/10 | Consistent Skill Contract format; four hooks still pending one (see above). |
180
- | **Agent Compatibility** | 9/10 | Full hard-mechanism support on Claude Code; verified advisory-text fallback on the other five platforms via `docs/architecture.md` and `scripts/installer.js`. |
181
- | **Beginner Friendliness** | 7.5/10 | Quick Start is genuinely 10 seconds, but Tier Routing / Rule of 3 / session-scoped state are load-bearing concepts a newcomer has to absorb before the system's behavior makes sense. Not addressed this round. |
182
-
183
- **What changed this cycle:** VERIFICATION.md §2's mechanism checks are now automated (`npm run test:mechanism`), VERIFICATION.md's own test recipes were rewritten to be Windows-safe (`node -e` stdin instead of a shell `echo` pipe), and `todo-cli.js` writes its shared state file atomically. Full detail in the commit history.
184
-
185
- ---
186
-
187
- ## 🤝 For Contributors
188
-
189
- To contribute to Harness or modify any Skill behavior, ensure you run the local self-regression suite first:
190
-
191
- ```bash
192
- # Run full hermetic static syntax & routing simulations
193
- npm run self-regression
194
- ```
195
-
196
- All script modifications must pass 100% cleanly before pushing to keep the runtime immunized against behavioral regression.
197
-
1
+ # Harness (Behavior Layer for AI Coding Agents)
2
+
3
+ [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
4
+
5
+ Harness is a lightweight, local behavior and orchestration runtime that wraps around your AI development sessions (Claude Code, Cursor, Copilot Chat, Codex, Continue.dev, Hermes Agent). It provides reactive hooks, routing boundaries, and circuit breakers designed to prevent infinite trial-and-error loops, costly over-engineering, and "lost-in-the-middle" context drift.
6
+
7
+ ---
8
+
9
+ ## The Problem
10
+
11
+ AI coding agents are highly capable, but they struggle with self-regulation, environment awareness, and attention limits:
12
+ 1. **The Infinite Retry Loop:** When an agent encounters a subtle compilation or test failure, its default behavior is to make micro-adjustments repeatedly (tweak and run, tweak and run) until it exhausts your token budget.
13
+ 2. **Environment Blindness:** Agents often assume standard Unix environments, hallucinating shell commands and paths when running on Windows, PowerShell, or sandboxed environments.
14
+ 3. **Lost-in-the-Middle Bloat:** As sessions grow, agents aggressively read too many large files or generate massive console logs, causing severe context degradation and reasoning hallucinations.
15
+
16
+ ---
17
+
18
+ ## Why Harness?
19
+
20
+ Harness acts as an automated system supervisor. It remains completely silent and out of the way, intervening only when execution boundaries are violated or failures are detected.
21
+
22
+ ### Comparison: Prompt vs. Skill vs. Harness
23
+
24
+ | Dimension | Prompt-Only (Custom Instructions) | Skill-Only (Task Guides) | Harness (Behavior Layer) |
25
+ |---|---|---|---|
26
+ | **Activation** | Always loaded (wastes prompt space) | Loaded on demand (requires manual trigger) | Reacts dynamically via native lifecycle hooks |
27
+ | **Fail-Safe** | No protection (model keeps retrying) | No protection (loops until token limit) | **Circuit Breaker:** Halts execution after 3 failures and alerts human |
28
+ | **Context Aware** | High risk of lost-in-the-middle bloat | Manages scope manually | **Bloat Shield:** Proactively audits diff sizes and logs warning alerts |
29
+ | **System Audit** | Blindly assumes shell syntax | Requires manual shell check | **Preflight:** Proactively detects Windows/Unix paths, shell type, and package manager |
30
+ | **Memory** | Resets on every new chat session | Static text rules | **Continuous Persistence:** Writes Write-Ahead Logs (WAL) for session recovery and immunizes workspace rules |
31
+
32
+ The "Harness" column above is Claude Code's behavior. On Cursor, Copilot, Codex, Continue.dev, and Hermes Agent — platforms with no hook/exit-code execution mechanism — Harness can only inject advisory text, which lands in the **Prompt-Only** column instead. See [Supported AI IDEs & Tools](#supported-ai-ides--tools) below.
33
+
34
+ ### When should I use Harness?
35
+ * You regularly use agentic coding tools (like Claude Code, Cursor, or Copilot) on medium-to-large codebases.
36
+ * You develop on Windows or in mixed shells (Git Bash, WSL, PowerShell) where agents frequently get shell syntax wrong.
37
+ * You want automated test-driven development (TDD) enforcement and safety guards to save token budgets.
38
+
39
+ ### When should I NOT use Harness?
40
+ * You only use chat interfaces for general questions without letting the AI run local commands or modify files.
41
+ * Your project has no test suite, or you prefer unconstrained, free-form agent generation.
42
+
43
+ ---
44
+
45
+ ## ⚡ Quick Start (Get Protected in 10s)
46
+
47
+ Harness integrates directly into your workspace. There is no heavy daemon, no paid external APIs, and zero configuration required.
48
+
49
+ ```bash
50
+ # Option A: Claude Code plugin (marketplace manifest included)
51
+ # /plugin marketplace add dyphn1/Harness-everything
52
+ # /plugin install harness-everything
53
+
54
+ # Option B: install Harness hooks and skills into your current workspace
55
+ npx github:dyphn1/Harness-everything install
56
+ ```
57
+
58
+ ### Expected Behavior After Installation:
59
+ 1. **Hook Registration:** Harness registers native hooks (e.g., inside `.claude/settings.json` for Claude Code) to intercept session starts and tool use.
60
+ 2. **Preflight Audit:** At session startup, a lightweight preflight script runs, printing a diagnostic environment block that tells the agent your exact OS, active shell, and package manager.
61
+ 3. **Guard Active:** The circuit breaker and context compactors are active in the background, consuming zero overhead unless triggered.
62
+
63
+ ### What Gets Installed (and How to Remove It)
64
+
65
+ The installer only ever writes to your workspace (or, with `--global`, your home directory) — no daemons, no registry entries, no network services. Depending on which platforms you select, it creates:
66
+
67
+ | File / Directory | Purpose |
68
+ |---|---|
69
+ | `.claude/settings.json` (merged) + `.claude/skills/` + `.claude/agents/` | Claude Code lifecycle hooks, project skills, and named Fable agents |
70
+ | `.cursorrules` | Cursor advisory rules |
71
+ | `.github/copilot-instructions.md` | Copilot Chat advisory instructions |
72
+ | `AGENTS.md` | Codex advisory instructions |
73
+ | `.continue/rules/harness.md` | Continue.dev advisory rules |
74
+ | `.hermes.md` | Hermes Agent advisory context |
75
+ | `.claude/harness-everything/` (or the per-platform equivalent) | Harness state: `manifest.json`, WAL, circuit-breaker counters |
76
+
77
+ The installer records the state directories in `.git/info/exclude` — a local-only git ignore file — so Harness state never lands in a commit and your working tree (including `.gitignore`) is never modified. Everything is removed with the built-in uninstaller:
78
+
79
+ ```bash
80
+ npx github:dyphn1/Harness-everything uninstall # interactive
81
+ npx github:dyphn1/Harness-everything uninstall --local --skills -y # non-interactive, workspace only
82
+ npx github:dyphn1/Harness-everything uninstall --global # also remove ~/.agents etc.
83
+ ```
84
+
85
+ ---
86
+
87
+ ## Visualizing the Flow
88
+
89
+ ### Without Harness (Endless Trial-and-Error Loop)
90
+ ```mermaid
91
+ flowchart TD
92
+ U([User Request]) --> A[AI Coding Agent]
93
+ A -->|Command/Edit| Env[Workspace Environment]
94
+ Env -->|Error / Failure| A
95
+ A -->|Tweak & Retry 1| Env
96
+ Env -->|Error / Failure| A
97
+ A -->|Tweak & Retry 2| Env
98
+ Env -->|Error / Failure| A
99
+ A -->|Tweak & Retry 3... N| Env
100
+ style A fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
101
+ ```
102
+
103
+ ### With Harness (Guarded and Routed Execution)
104
+ ```mermaid
105
+ flowchart TD
106
+ U([User Request]) --> H_Router[Harness Router]
107
+ H_Router -->|Tier 1: Trivial| T1[Direct Edit]
108
+ H_Router -->|Tier 2: Standard| T2["Native TODO / Markdown checklist"]
109
+ H_Router -->|Tier 3: Macro| T3[Fable Multi-Agent Flow]
110
+
111
+ T2 & T3 --> Exec[Execute Code / Run Commands]
112
+ Exec --> Gate{verify-gate.js}
113
+
114
+ Gate -->|Exit 1: Fix & retry| Exec
115
+ Gate -->|Exit 1 - Repeated| CB{Circuit Breaker rule-of-3.js}
116
+ CB -->|Fails 3x| ZO[Zoom Out: Ask Human / Reflect]
117
+ ZO -->|Fresh Diagnosis| Exec
118
+ Gate -->|Exit 0: Success| Done[Checklist item complete]
119
+
120
+ Done --> SE[Self-Evolve: Update Rules]
121
+ style H_Router fill:#c8e6c9,stroke:#2e7d32,stroke-width:1px,color:#000000
122
+ style CB fill:#fff9c4,stroke:#fbc02d,stroke-width:1px,color:#000000
123
+ style ZO fill:#ffcc80,stroke:#ef6c00,stroke-width:1px,color:#000000
124
+ style Gate fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
125
+ ```
126
+
127
+ ---
128
+
129
+ ## Core Modules & Concepts
130
+
131
+ Harness operates through six core cognitive concepts:
132
+
133
+ 1. **Router (`tier-router.js`):** Prevents over-engineering. Triages incoming tasks into Tiers: Tier 1 (Direct Edit, no plans), Tier 2 (Standard TDD enforcement), or Tier 3 (Macro Multi-Agent planning and delegation). It also scans every installed platform's `manifest.json` for skills `self-evolve` has generated dynamically, and auto-surfaces the ones whose keyword triggers match the current prompt — so a lesson learned in one session gets suggested again without the human having to remember it exists. If nothing matches at all — including nothing already kept from the open skills ecosystem — `find-skills` checks `npx skills list` live and, if still nothing, searches `skills.sh`/`npx skills`. With explicit approval it defaults to applying the result ephemerally (a content-addressed OS-temp cache, zero footprint after the OS reclaims it) and only permanently installs via `npx skills add` when the user says they'll reuse it — neither path is cached into `manifest.json` the way `generated[]` is, since third-party content isn't lifecycle-owned by Harness.
134
+ 2. **Guard (`rule-of-3.js`):** The fail-safe circuit breaker. Tracks failure signatures across terminal runs. If a test or command fails 3 times with the same signature, it locks mutating tools and forces a `zoom-out` reflection: re-verify every assumption with read-only tools, write a fact-checked report, then resume on a fresh diagnosis. The human partner is pulled in only for genuine decisions — or when the same signature trips the breaker a second time. A companion `Stop` gate (`stop-gate.js`) bounces the end of a turn once per edit batch when edits were never followed by a successful verification command.
135
+ 3. **Memory (`state-persist.js`):** Session transaction logging. Stores a local Write-Ahead Log (WAL) of milestones, preventing agents from forgetting their current task state if a session limits out or restarts.
136
+ 4. **Reflection (`self-evolve`):** Long-term workspace immunization. Upon task completion, the agent reflects on the root cause of resolved issues, then judges whether the lesson is a simple rule or a reusable, complex pattern: simple rules are appended to local workspace rules (`RULES.md`); genuinely reusable patterns are instead packaged as a dynamic skill (via `skill-creator`'s Dynamic Skill Generation Contract) and registered in `manifest.json` so the Router picks it up in future sessions. Either path is validated by a hermetic self-regression suite before it's persisted.
137
+ 5. **Subagent Scope Guard (`subagent-scope-guard.js`):** Diffs the whole repo's `git status` before and after every subagent (`Task`) burst, not just the files it was briefed to touch. Catches a subagent that was told to only read/verify but edited files anyway — a real failure mode, not a hypothetical one.
138
+ 6. **Cognitive Laws (Agent Cognitive OS):** Six governing laws — Intent Precedence, State Handoff Awakening, Elimination & Prediction, Evidence Assertion, Adversarial Falsification, and Code-Documentation Alignment — are woven directly into the specific skill phase each one governs (e.g. Evidence Assertion inside `tdd`'s RED/GREEN/REFACTOR gates, State Handoff Awakening inside `zoom-out`'s reflection phase) rather than duplicated in one shared file. See the relevant skill's own `SKILL.md` for where a given law actually applies.
139
+
140
+ ---
141
+
142
+ ## Supported AI IDEs & Tools
143
+
144
+ **Only Claude Code gets the hard-boundary hooks.** Every other platform below has no hook/exit-code execution mechanism, so `harness-everything` can only inject advisory text — same protection level as the "Prompt-Only" column in the comparison table above. There is no circuit breaker, no preflight audit, and no WAL on those platforms unless Claude Code (or another hook-capable tool) is also driving the same repo.
145
+
146
+ | AI Agent Tool | Integration Method | Local Target Location | Enforcement |
147
+ |---|---|---|---|
148
+ | **Claude Code** | Native Lifecycle Hooks (`PreToolUse`, `PostToolUse`, `SessionStart`) | `.claude/settings.json` (project) / `~/.claude/settings.json` (user)<br>*.claude/skills/* (Project Skills) / *~/.claude/skills/* (Global Skills) | **Hard** — hooks can block a tool call (`exit(2)`) |
149
+ | **Cursor** | Native Project Rules | `.cursorrules` | Advisory only |
150
+ | **Copilot Chat** | Custom Instructions | `.github/copilot-instructions.md` | Advisory only |
151
+ | **Codex** | Custom Instructions (`AGENTS.md`, not `.codex/config.toml` — that file controls CLI/sandbox behavior, not prompt content) | `AGENTS.md` | Advisory only |
152
+ | **Continue.dev** | Native project rules (a dedicated Markdown file with YAML frontmatter, `alwaysApply: true`) | `.continue/rules/harness.md` (project) / `~/.continue/rules/harness.md` (user, via `--global`) | Advisory only |
153
+ | **Hermes Agent** ([Nous Research](https://hermes-agent.nousresearch.com/)) | Auto-loaded project context file (Hermes also reads `AGENTS.md`/`CLAUDE.md`/`.cursorrules` from the same directory if present, truncated at ~20k chars) | `.hermes.md` (project only — Hermes has no documented global project-instructions equivalent, so `--global --hermes` is a documented no-op) | Advisory only |
154
+
155
+ ---
156
+
157
+ ## Repository Index
158
+
159
+ ## Multi-Agent Workspace
160
+
161
+ Use the canonical `multi-agent-workspace` skill for permanent multi-agent
162
+ infrastructure:
163
+
164
+ ```bash
165
+ node multi-agent-workspace/scripts/scaffold.js --workspace . \
166
+ --agency-source <path-to-agency-agents> --division engineering --platform codex
167
+ ```
168
+
169
+ The source is read-only input. The generated `.harness/multi-agent/` directory
170
+ records the six zones, selected metadata, supported converter targets,
171
+ launcher, immutable router, memory index, and structured handoff. Omit the
172
+ source for an explicit unavailable-catalog fallback; do not treat it as a
173
+ complete roster.
174
+
175
+ This repo uses a flat layout (waza/agentskills.io convention). The table below maps each top-level directory to its role.
176
+
177
+ | Directory | Category | Description |
178
+ |---|---|---|
179
+ | `harness-everything` | **Core Runtime** | Bootstrap, tier-router, verify-gate, self-heal |
180
+ | `hooks` | **Core Runtime** | Claude Code lifecycle hooks (circuit breaker, scope guard, stop gate, etc.) |
181
+ | `scripts` | **Core Runtime** | Installer, manifest, prompts, workspace utilities |
182
+ | `bin` | **Core Runtime** | `harness` CLI entry point |
183
+ | `ci` | **Quality Gates** | Consistency check, description collision, mechanism tests, negative controls |
184
+ | `.github` | **CI/CD** | GitHub Actions workflows (ci.yml, release.yml, behavioral-evals.yml) |
185
+ | `.claude-plugin` | **Distribution** | Plugin manifests for Claude Code marketplace |
186
+ | `evals` | **Routing Evals** | 26 trigger/routing eval suites (waza format) |
187
+ | `behavioral-evals` | **Behavioral Evals** | LLM-level discipline cases (headless agent sessions) |
188
+ | `benchmarks` | **Benchmarks** | BENCHMARK_SOP fixtures and recorded A/B results |
189
+ | `docs` | **Documentation** | Philosophy, architecture, routing, reflection, audit |
190
+ | `references` | **Documentation** | Shared checklists (security, performance, definition-of-done) |
191
+ | `multi-agent-workspace` | **Skill (Tier 3)** | Scaffold six zones, select agency specialists, and generate bounded launchers |
192
+ | `environment-detection` | **Foundation** | Preflight: detect OS, shell, package manager |
193
+ | `eval-harness` | **Skill (Tier 2)** | Evaluate agent outputs against rubrics |
194
+ | `fable-discipline` | **Skill (Tier 3)** | Enforce Fable execution guardrails |
195
+ | `fable-mode` | **Skill (Tier 3)** | Multi-agent orchestration with milestone gates |
196
+ | `find-skills` | **Meta** | Discover and install skills from open ecosystems |
197
+ | `git-commit` | **Skill (Tier 1)** | Conventional commit messages with verification |
198
+ | `grill-me` | **Skill (Tier 2)** | Adversarial plan interrogation before implementation |
199
+ | `grill-with-docs` | **Skill (Tier 3)** | Domain-model and decision alignment before design publication |
200
+ | `improve-codebase-architecture` | **Skill (Tier 2)** | Architectural refactoring with evidence |
201
+ | `install-cognitive-os` | **Foundation** | Install Harness cognitive OS into workspace |
202
+ | `repo-docs` | **Skill (Tier 3)** | Generate repository documentation |
203
+ | `rewrite-commits` | **Skill (Tier 1)** | Interactive rebase and commit history cleanup |
204
+ | `security-review` | **Skill (Tier 2)** | OWASP/STRIDE security review |
205
+ | `self-evolve` | **Skill (Tier 2)** | Workspace immunization via dynamic skills |
206
+ | `skill-creator` | **Meta** | Create new skills from patterns |
207
+ | `skill-style` | **Meta** | Skill authoring style guide |
208
+ | `tdd` | **Skill (Tier 2)** | Test-driven development enforcement |
209
+ | `to-spec` | **Advisory (Tier 2/3)** | Publish specs from settled conversations |
210
+ | `to-tickets` | **Advisory (Tier 2/3)** | Decompose settled specs into tracked tickets |
211
+ | `todo-driven-workflow` | **Foundation** | Native TODO or Markdown checklist discipline |
212
+ | `using-git-worktrees` | **Skill (Tier 2)** | Git worktree concurrency patterns |
213
+ | `verification-loop` | **Skill (Tier 2)** | Verify-before-complete enforcement |
214
+ | `verify-before-claim` | **Always-on discipline** | Fact-audit before asserting claims |
215
+ | `zoom-out` | **Circuit breaker** | Circuit-breaker reflection protocol |
216
+ | `opencode-plugin` | **Platform Plugin** | Enforcement hooks for opencode (post-edit, pre-complete, circuit-breaker) |
217
+
218
+ ---
219
+
220
+ ## Deeper Documentation
221
+
222
+ For a deep dive into individual modules and the underlying philosophy, explore our sub-documents:
223
+
224
+ * [Harness Philosophy](docs/philosophy.md): The core behavior-first, intervention-only design.
225
+ * [Harness Architecture](docs/architecture.md): Lifecycle hooks, security model, and data locality.
226
+ * [Harness Routing & Triage](docs/routing.md): Detailed trigger criteria for Tiers 1, 2, and 3.
227
+ * [Harness Reflection & Memory](docs/reflection.md): WAL session handoffs and workspace rules immunization.
228
+ * [Harness Audit Log](docs/audit.md): Dated self-audit scorecards, methodology, and per-cycle change log.
229
+
230
+ Fable model selection is documented in [fable-mode/references/model-matrix.md](fable-mode/references/model-matrix.md); the explicit entrypoints are `fable-haiku`, `fable-sonnet`, and `fable-opus`.
231
+
232
+ ---
233
+
234
+ ## Benchmarks & Testing
235
+
236
+ **If you are an agent asked to verify a Harness install, start at [VERIFICATION.md](VERIFICATION.md), not here.** It gives exact commands with exact expected output — install artifact checks for every platform, mechanism-level checks (Claude Code only), the behavioral test prompts below, and an acceptance scorecard to fill in. Do not report "it works" from reading the code — every check there names a command to actually run.
237
+
238
+ `npm test` (`self-evolve/scripts/self-regression.js`) runs deterministic syntax, CLI, routing-matrix, positive skill-route coverage, reference, behavioral-case, Fable model-mode, and mechanism checks (`ci/mechanism-test.js`, `npm run test:mechanism` to run it alone). The suite checks real exit codes and stderr, not just "the code looks right." It runs in CI on every push and pull request (`.github/workflows/ci.yml`); live model evaluations remain explicitly on-demand.
239
+
240
+ For a fuller vanilla-vs-Harness behavioral comparison, see [Harness Skills Benchmark SOP](BENCHMARK_SOP.md) — standardized, reproducible scenarios:
241
+ * **Test A:** Over-engineering defense (Tier 1 typo correction)
242
+ * **Test B:** Micro-error loop defense (Tier 2 bug resolution)
243
+ * **Test C:** Attention loss and hallucination (Tier 3 module refactoring)
244
+ * **Test D:** Knowledge boundary constraints (Offline hallucination prevention)
245
+ * **Test E:** Terminal environment and shell awareness (Windows/Unix shell detection)
246
+ * **Test F** (in VERIFICATION.md, not BENCHMARK_SOP.md): fact-audit discipline — does the agent verify an external-behavior claim before asserting it?
247
+
248
+ Benchmark **results** are tracked in [benchmarks/](benchmarks/README.md) (`run.js scaffold` builds the fixture, `record` commits a schema-validated result bound to a session log). Until those cells are filled, effectiveness claims are unbacked by recorded evidence.
249
+
250
+ ### Behavioral evals (LLM-level, on demand)
251
+
252
+ Mechanism tests prove the hooks enforce gates; only live sessions prove agents follow the disciplines. [behavioral-evals/](behavioral-evals/README.md) runs discipline cases (including pressure variants like "we ship in 5 minutes, skip checks") against headless agent sessions (`claude -p`, opencode) in throwaway workspaces: `npm run eval:behavioral`. Token-costing by design — never wired into CI.
253
+
254
+ ### Catalog hygiene
255
+
256
+ `npm run test:consistency` keeps the distribution manifests, docs links, skill frontmatter, and routing-eval coverage in lockstep with what is actually on disk; `npm run test:references` checks every executable/deep-dive path named by `SKILL.md`; `npm run test:release` compares the skill catalog and references with `v0.3.3-beta`; and `harness verify-install` detects stale installed versions or file trees. `npm run test:collision` fails CI when two skills' descriptions overlap enough to confuse the router. The repository checks run on every push (`.github/workflows/ci.yml`).
257
+
258
+ ---
259
+
260
+ ## 📊 System Evaluation
261
+
262
+ Harness audits itself on a dated cycle by running its own test suite and VERIFICATION.md recipes — never by reading the code and assuming it works. The full scorecards, methodology, and per-cycle change log live in [docs/audit.md](docs/audit.md).
263
+
264
+ **Latest local baseline — 2026-09-01**: 26/26 on-disk skills, 26/26 routing-eval directories, and the local npm gates are green. Waza remains a CI-only gate in this checkout because its installer is not available as an npm package; run it on an LF-normalized export as documented by CI.
265
+
266
+ Measure on an LF export, not a Windows working tree — CRLF can inflate waza's token counts and trigger false budget failures.
267
+
268
+ ---
269
+
270
+ ## 🤝 For Contributors
271
+
272
+ To contribute to Harness or modify any Skill behavior, ensure you run the local self-regression suite first:
273
+
274
+ ```bash
275
+ # Run full hermetic static syntax & routing simulations
276
+ npm run self-regression
277
+ ```
278
+
279
+ All script modifications must pass 100% cleanly before pushing to keep the runtime immunized against behavioral regression.
package/SECURITY.md ADDED
@@ -0,0 +1,33 @@
1
+ # Security Policy
2
+
3
+ ## Reporting a Vulnerability
4
+
5
+ If you discover a security vulnerability in Harness, please report it responsibly.
6
+
7
+ **Do not open a public issue for security vulnerabilities.**
8
+
9
+ Instead, email: dyphn1@hotmail.com
10
+
11
+ Include:
12
+ - Description of the vulnerability
13
+ - Steps to reproduce
14
+ - Potential impact
15
+
16
+ ## Response Timeline
17
+
18
+ - Acknowledgment within 48 hours
19
+ - Fix or mitigation plan within 7 days for confirmed vulnerabilities
20
+
21
+ ## Scope
22
+
23
+ This policy covers the Harness-everything repository and its published npm package.
24
+
25
+ ## Hooks and Runtime Behavior
26
+
27
+ Harness installs execution hooks (circuit breaker, scope guard, stop gate) into Claude Code sessions. These hooks:
28
+
29
+ - Enforce file-scope boundaries
30
+ - Track verification state
31
+ - Block unsafe operations
32
+
33
+ If you find a bypass or unintended behavior in these mechanisms, please report it.