harness-everything 0.3.6 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +20 -0
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +8 -7
- package/.github/workflows/ci.yml +23 -18
- package/.github/workflows/release.yml +53 -27
- package/.releaserc.json +113 -0
- package/AGENTS.md +34 -23
- package/BENCHMARK_SOP.md +38 -18
- package/CHANGELOG.md +194 -3
- package/CONTRIBUTING.md +17 -4
- package/PRIVACY.md +25 -0
- package/README.md +118 -87
- package/RELEASING.md +82 -0
- package/SUPPORT.md +27 -0
- package/TERMS.md +21 -0
- package/VERIFICATION.md +175 -286
- package/behavioral-evals/README.md +21 -0
- package/behavioral-evals/case-validator.js +194 -0
- package/behavioral-evals/cases/baseline-environment-detection.yaml +21 -0
- package/behavioral-evals/cases/baseline-eval-harness.yaml +27 -0
- package/behavioral-evals/cases/baseline-fable-discipline.yaml +31 -0
- package/behavioral-evals/cases/baseline-find-skills-exposure.yaml +49 -0
- package/behavioral-evals/cases/baseline-find-skills.yaml +21 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +2 -2
- package/behavioral-evals/cases/baseline-install-cognitive-os.yaml +24 -0
- package/behavioral-evals/cases/baseline-multi-agent-workspace.yaml +28 -0
- package/behavioral-evals/cases/baseline-rewrite-commits.yaml +34 -0
- package/behavioral-evals/cases/baseline-self-evolve.yaml +31 -0
- package/behavioral-evals/cases/baseline-skill-creator.yaml +20 -0
- package/behavioral-evals/cases/baseline-skill-style.yaml +21 -0
- package/behavioral-evals/cases/baseline-to-spec.yaml +34 -0
- package/behavioral-evals/cases/baseline-to-tickets.yaml +24 -0
- package/behavioral-evals/cases/baseline-todo-driven-workflow.yaml +32 -0
- package/behavioral-evals/cases/baseline-using-git-worktrees.yaml +15 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +2 -2
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +2 -2
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +7 -2
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +7 -4
- package/behavioral-evals/cases/scope-discipline.yaml +2 -1
- package/behavioral-evals/cases/verify-before-done.yaml +7 -5
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/historical-case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/live-run-blocker.md +16 -0
- package/behavioral-evals/evidence/2026-09-07/manifest.json +88 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/historical-case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/historical-case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/case.yaml +30 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/historical-case.yaml +23 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.json +426 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.md +24 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/case.yaml +19 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/result.sanitized.json +480 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/transcript.sanitized.jsonl +95 -0
- package/behavioral-evals/evidence-tool.js +443 -0
- package/behavioral-evals/fixtures/claude-denied.jsonl +3 -0
- package/behavioral-evals/fixtures/claude-legacy.json +1 -0
- package/behavioral-evals/fixtures/claude-oral-only.jsonl +2 -0
- package/behavioral-evals/fixtures/claude-success.jsonl +5 -0
- package/behavioral-evals/fixtures/claude-truncated.jsonl +3 -0
- package/behavioral-evals/fixtures/opencode-success.jsonl +6 -0
- package/behavioral-evals/results/2026-08-31-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-09-03-baseline-environment-detection-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-eval-harness-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-fable-discipline-pair.json +149 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-exposure-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-install-cognitive-os-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-multi-agent-workspace-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-rewrite-commits-pair.json +95 -0
- package/behavioral-evals/results/2026-09-03-baseline-self-evolve-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-creator-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-style-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-spec-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-tickets-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-todo-driven-workflow-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-using-git-worktrees-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-summary-both.json +41 -0
- package/behavioral-evals/results/2026-09-04-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-04-baseline-self-evolve-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-baseline-skill-creator-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-summary-both.json +41 -0
- package/behavioral-evals/run-with-plugin.js +15 -31
- package/behavioral-evals/run.js +446 -105
- package/behavioral-evals/transcript-parser.js +591 -0
- package/bin/cli.js +24 -12
- package/ci/ab-test-cases/grill-me-adversarial.yaml +14 -4
- package/ci/ab-test-cases/tdd-test-first.yaml +13 -6
- package/ci/ab-test-cases/verification-loop-gates.yaml +20 -7
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +7 -6
- package/ci/ab-test-harness.js +113 -38
- package/ci/action-gate-platform-doc.test.js +31 -0
- package/ci/action-gate.test.js +260 -0
- package/ci/cases/case2.json +2 -2
- package/ci/consistency-check.js +51 -5
- package/ci/description-collision.js +15 -6
- package/ci/disclosure-check.js +225 -0
- package/ci/doc-capability-consistency.test.js +180 -0
- package/ci/fixtures/phase4/parallel-disjoint.stages.json +32 -0
- package/ci/fixtures/phase4/parallel-read-only.stages.json +32 -0
- package/ci/installer-roundtrip.test.js +159 -0
- package/ci/invariant-routing.test.js +74 -0
- package/ci/issue99-runtime.test.js +218 -0
- package/ci/mechanism-2aa-codex-action-gate-policy.test.js +117 -0
- package/ci/mechanism-2ab-openai-plugin-sync-integrity.test.js +80 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +43 -1
- package/ci/mechanism-2k-reference-check.test.js +57 -2
- package/ci/mechanism-2l-multi-agent-workspace.test.js +202 -6
- package/ci/mechanism-2m-self-repo-guard.test.js +65 -0
- package/ci/mechanism-2n-opencode-plugin.test.js +461 -0
- package/ci/mechanism-2o-state-home-isolation.test.js +183 -0
- package/ci/mechanism-2p-installer-links.test.js +183 -0
- package/ci/mechanism-2q-installer-global-shared-store.test.js +132 -0
- package/ci/mechanism-2r-uninstall-flags.test.js +148 -0
- package/ci/mechanism-2s-tdd-quality.test.js +203 -0
- package/ci/mechanism-2t-disclosure.test.js +90 -0
- package/ci/mechanism-2t-evidence-triage.test.js +200 -0
- package/ci/mechanism-2t-issue-20.test.js +75 -0
- package/ci/mechanism-2t-platform-skill-targets.test.js +164 -0
- package/ci/mechanism-2t-transcript-parser.test.js +179 -0
- package/ci/mechanism-2t-workspace-identity.test.js +200 -0
- package/ci/mechanism-2u-issue-42-regressions.test.js +239 -0
- package/ci/mechanism-2u-issue-44.test.js +35 -0
- package/ci/mechanism-2v-verifier-boundaries.test.js +138 -0
- package/ci/mechanism-2w-platform-compatibility.test.js +148 -0
- package/ci/mechanism-2x-release-automation.test.js +157 -0
- package/ci/mechanism-2x-release-commit-message.test.js +66 -0
- package/ci/mechanism-2y-release-tooling-compat.test.js +47 -0
- package/ci/mechanism-2z-claude-plugin-manifest.test.js +29 -0
- package/ci/mutation-check.js +53 -0
- package/ci/mutation-loader.js +57 -0
- package/ci/negative-controls.js +76 -46
- package/ci/openai-plugin-package.test.js +182 -0
- package/ci/openai-plugin-runtime.test.js +175 -0
- package/ci/openai-plugin-submission.test.js +101 -0
- package/ci/phase4-ensemble-memory.test.js +307 -0
- package/ci/reference-check.js +73 -25
- package/ci/release-consistency-check.js +24 -1
- package/ci/router-phase2-safety.test.js +81 -0
- package/ci/router-workflow-plan.test.js +264 -0
- package/ci/routing-check.js +6 -5
- package/ci/runner.js +6 -5
- package/ci/skill-routing-check.js +48 -11
- package/ci/test-behavioral-case-validation.js +62 -0
- package/ci/test-helper.js +10 -0
- package/ci/workflow-plan-consumers-edge.test.js +72 -0
- package/ci/workflow-plan-consumers.test.js +276 -0
- package/docs/architecture.md +137 -57
- package/docs/audit.md +22 -20
- package/docs/codex-action-gate.md +90 -0
- package/docs/issue-20-rollup.md +47 -0
- package/docs/mechanism-first-skill-mesh.md +22 -20
- package/docs/openai-plugin.md +118 -0
- package/docs/philosophy.md +28 -17
- package/docs/platform-capabilities.md +73 -0
- package/docs/platform-compatibility.json +270 -0
- package/docs/reflection.md +37 -22
- package/docs/release-evidence.md +25 -0
- package/docs/routing.md +157 -44
- package/docs/security-audit.md +38 -0
- package/docs/troubleshooting.md +82 -46
- package/docs/verifier-boundaries.md +45 -0
- package/docs/workflows/fable-mode.md +77 -21
- package/docs/workflows/find-skills.md +40 -0
- package/docs/workflows/grill-with-docs.md +2 -2
- package/docs/workflows/multi-agent-workspace.md +22 -10
- package/docs/workflows/self-evolve.md +33 -0
- package/docs/workflows/tdd.md +36 -0
- package/docs/workflows/todo-driven-workflow.md +33 -0
- package/environment-detection/SKILL.md +3 -3
- package/eval-harness/SKILL.md +7 -7
- package/eval-harness/scripts/evaluate.js +12 -10
- package/evals/evaluation-reports.md +14 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +1 -1
- package/evals/harness-everything/tasks/positive-tier2.yaml +1 -1
- package/evals/harness-everything/tasks/positive-tier3.yaml +1 -1
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +1 -1
- package/fable-discipline/SKILL.md +2 -2
- package/fable-mode/CONTRACT-FORMAT.md +147 -33
- package/fable-mode/SKILL.md +3 -3
- package/fable-mode/agents/fable-orchestrator.md +95 -35
- package/fable-mode/execution-guardrails/SKILL.md +2 -2
- package/fable-mode/fable-haiku/SKILL.md +3 -3
- package/fable-mode/fable-opus/SKILL.md +3 -3
- package/fable-mode/fable-sonnet/SKILL.md +3 -3
- package/fable-mode/scripts/ensemble-review.js +371 -0
- package/fable-mode/scripts/model-selector.js +134 -3
- package/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/find-skills/SKILL.md +5 -5
- package/find-skills/references/discovery-flow.md +1 -1
- package/find-skills/scripts/use-skill.js +7 -1
- package/git-commit/SKILL.md +4 -11
- package/grill-me/SKILL.md +5 -5
- package/grill-with-docs/SKILL.md +7 -7
- package/grill-with-docs/references/session-playbook.md +6 -4
- package/harness-everything/SKILL.md +23 -22
- package/harness-everything/references/router-workflow-plan.md +128 -0
- package/harness-everything/references/triage-and-tiers.md +173 -96
- package/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/harness-everything/scripts/bootstrap.js +13 -4
- package/harness-everything/scripts/ensemble-policy.js +102 -0
- package/harness-everything/scripts/kernel-router.js +212 -0
- package/harness-everything/scripts/router-contract.js +612 -0
- package/harness-everything/scripts/self-heal.js +29 -20
- package/harness-everything/scripts/tier-router.js +346 -221
- package/hooks/hooks.json +60 -15
- package/hooks/scripts/action-gate-rules.json +77 -0
- package/hooks/scripts/action-gate.js +528 -0
- package/hooks/scripts/atomic-commit-check.js +5 -5
- package/hooks/scripts/codex-action-gate-post.js +46 -0
- package/hooks/scripts/codex-permission-request.js +140 -0
- package/hooks/scripts/context-compact.js +4 -3
- package/hooks/scripts/contract-test.js +118 -51
- package/hooks/scripts/lib/fable-contracts.js +121 -0
- package/hooks/scripts/lib/harness-state.js +164 -28
- package/hooks/scripts/lib/platforms/claude.js +45 -62
- package/hooks/scripts/lib/platforms/codex.js +35 -61
- package/hooks/scripts/lib/platforms/continue.js +7 -7
- package/hooks/scripts/lib/platforms/copilot.js +29 -53
- package/hooks/scripts/lib/platforms/hermes.js +71 -20
- package/hooks/scripts/openai-session-start.js +81 -0
- package/hooks/scripts/rule-of-3-tracker.js +9 -8
- package/hooks/scripts/rule-of-3.js +29 -21
- package/hooks/scripts/state-persist.js +5 -4
- package/hooks/scripts/stop-gate.js +2 -2
- package/hooks/scripts/subagent-scope-guard.js +144 -38
- package/hooks/scripts/subagent-start.js +19 -0
- package/hooks/scripts/subagent-stop.js +6 -0
- package/improve-codebase-architecture/SKILL.md +2 -4
- package/improve-codebase-architecture/references/README.md +16 -0
- package/install-cognitive-os/SKILL.md +28 -22
- package/multi-agent-workspace/SKILL.md +16 -15
- package/multi-agent-workspace/references/architecture-guide.md +13 -9
- package/multi-agent-workspace/references/orchestration.md +39 -22
- package/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/multi-agent-workspace/scripts/index_memory.js +79 -26
- package/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/multi-agent-workspace/scripts/scaffold.js +258 -103
- package/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/multi-agent-workspace/workflows/03-generation.md +8 -10
- package/multi-agent-workspace/workflows/04-launcher.md +5 -5
- package/opencode-plugin/README.md +114 -71
- package/opencode-plugin/hooks/verify.js +62 -12
- package/opencode-plugin/index.mjs +547 -0
- package/opencode-plugin/plugin.json +4 -12
- package/package.json +16 -3
- package/plugins/harness-everything/.codex-plugin/plugin.json +39 -0
- package/plugins/harness-everything/hooks/hooks.json +197 -0
- package/plugins/harness-everything/hooks/scripts/action-gate-rules.json +77 -0
- package/plugins/harness-everything/hooks/scripts/action-gate.js +528 -0
- package/plugins/harness-everything/hooks/scripts/atomic-commit-check.js +68 -0
- package/plugins/harness-everything/hooks/scripts/boundary-guard.js +55 -0
- package/plugins/harness-everything/hooks/scripts/codex-action-gate-post.js +46 -0
- package/plugins/harness-everything/hooks/scripts/codex-permission-request.js +140 -0
- package/plugins/harness-everything/hooks/scripts/context-compact.js +115 -0
- package/plugins/harness-everything/hooks/scripts/contract-test.js +146 -0
- package/plugins/harness-everything/hooks/scripts/depth-guard.js +65 -0
- package/plugins/harness-everything/hooks/scripts/lib/fable-contracts.js +121 -0
- package/plugins/harness-everything/hooks/scripts/lib/harness-state.js +275 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/claude.js +168 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/codex.js +88 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/continue.js +91 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/copilot.js +85 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/cursor.js +89 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/hermes.js +108 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/index.js +17 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/worktrees.js +20 -0
- package/plugins/harness-everything/hooks/scripts/lib/workspace.js +290 -0
- package/plugins/harness-everything/hooks/scripts/openai-session-start.js +81 -0
- package/plugins/harness-everything/hooks/scripts/reset-circuit-breaker.js +30 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3-tracker.js +125 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3.js +195 -0
- package/plugins/harness-everything/hooks/scripts/state-persist.js +100 -0
- package/plugins/harness-everything/hooks/scripts/stop-gate.js +80 -0
- package/plugins/harness-everything/hooks/scripts/subagent-scope-guard.js +194 -0
- package/plugins/harness-everything/hooks/scripts/subagent-start.js +19 -0
- package/plugins/harness-everything/hooks/scripts/subagent-stop.js +6 -0
- package/plugins/harness-everything/hooks/session-start.js +81 -0
- package/plugins/harness-everything/plugin.json +43 -0
- package/plugins/harness-everything/scripts/lib/advisory-text.js +174 -0
- package/plugins/harness-everything/skills/environment-detection/SKILL.md +41 -0
- package/plugins/harness-everything/skills/environment-detection/references/shell-syntax-rules.md +76 -0
- package/plugins/harness-everything/skills/environment-detection/scripts/preflight.js +103 -0
- package/plugins/harness-everything/skills/eval-harness/SKILL.md +42 -0
- package/plugins/harness-everything/skills/eval-harness/references/scoring-rubric.md +68 -0
- package/plugins/harness-everything/skills/eval-harness/scripts/evaluate.js +79 -0
- package/plugins/harness-everything/skills/fable-discipline/SKILL.md +41 -0
- package/plugins/harness-everything/skills/fable-discipline/references/discipline-rules.md +22 -0
- package/plugins/harness-everything/skills/fable-mode/CONTRACT-FORMAT.md +165 -0
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +159 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-verifier.md +32 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-haiku.md +52 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-sonnet.md +53 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +43 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/model-matrix.json +32 -0
- package/plugins/harness-everything/skills/fable-mode/references/execution-phases.md +37 -0
- package/plugins/harness-everything/skills/fable-mode/references/model-matrix.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/ensemble-review.js +371 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/model-selector.js +260 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/plugins/harness-everything/skills/find-skills/SKILL.md +48 -0
- package/plugins/harness-everything/skills/find-skills/references/discovery-flow.md +102 -0
- package/plugins/harness-everything/skills/find-skills/scripts/use-skill.js +112 -0
- package/plugins/harness-everything/skills/git-commit/SKILL.md +38 -0
- package/plugins/harness-everything/skills/git-commit/guides/ANGULAR_STYLE.md +61 -0
- package/plugins/harness-everything/skills/git-commit/guides/COMMIT_GENERATION.md +24 -0
- package/plugins/harness-everything/skills/git-commit/guides/LANGUAGE_DETECTION.md +16 -0
- package/plugins/harness-everything/skills/git-commit/guides/MAIN_REPO.md +15 -0
- package/plugins/harness-everything/skills/git-commit/guides/SUBMODULES.md +27 -0
- package/plugins/harness-everything/skills/git-commit/references/commit-flow.md +30 -0
- package/plugins/harness-everything/skills/grill-me/SKILL.md +43 -0
- package/plugins/harness-everything/skills/grill-me/references/grilling-playbook.md +38 -0
- package/plugins/harness-everything/skills/grill-with-docs/ADR-FORMAT.md +47 -0
- package/plugins/harness-everything/skills/grill-with-docs/CONTEXT-FORMAT.md +63 -0
- package/plugins/harness-everything/skills/grill-with-docs/SKILL.md +41 -0
- package/plugins/harness-everything/skills/grill-with-docs/references/session-playbook.md +99 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +43 -0
- package/plugins/harness-everything/skills/harness-everything/examples/README.md +9 -0
- package/plugins/harness-everything/skills/harness-everything/examples/todo-workflow.md +22 -0
- package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md +128 -0
- package/plugins/harness-everything/skills/harness-everything/references/skill-registry.md +39 -0
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +193 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/bootstrap.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/ensemble-policy.js +102 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/router-contract.js +612 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/routing-keywords.json +320 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/self-heal.js +250 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +459 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/verify-gate.js +100 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/SKILL.md +38 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/DEEPENING.md +37 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/HTML-REPORT.md +123 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/LANGUAGE.md +53 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/references/README.md +16 -0
- package/plugins/harness-everything/skills/install-cognitive-os/SKILL.md +44 -0
- package/plugins/harness-everything/skills/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/csharp-monorepo.md +123 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/typescript-cli.md +126 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-claude.md +77 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-codex.md +146 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-copilot.md +74 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/role-capabilities.md +108 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/agency-agents.md +31 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/architecture-guide.md +36 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/orchestration.md +65 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/index_memory.js +104 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/scaffold.js +408 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/README.md +17 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/agent-launcher.md +68 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-csharp.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-polyglot.md +62 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-ts.md +57 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/memory-keeper.md +37 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/requirement-analyzer.md +66 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/specialist-agent.md +34 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/task-verifier.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/tool-maker.md +33 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/01-init.md +22 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/03-generation.md +25 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/plugins/harness-everything/skills/repo-docs/SKILL.md +42 -0
- package/plugins/harness-everything/skills/repo-docs/references/process-guide.md +60 -0
- package/plugins/harness-everything/skills/repo-docs/templates/agents-template.md +264 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-agents-template.md +50 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-readme-template.md +83 -0
- package/plugins/harness-everything/skills/repo-docs/templates/manual-recon.md +117 -0
- package/plugins/harness-everything/skills/repo-docs/templates/multi-skills-readme-template.md +72 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-agents-template.md +78 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-readme-template.md +95 -0
- package/plugins/harness-everything/skills/repo-docs/templates/readme-template.md +136 -0
- package/plugins/harness-everything/skills/repo-docs/templates/single-skill-readme-template.md +70 -0
- package/plugins/harness-everything/skills/repo-docs/templates/skills-agents-template.md +60 -0
- package/plugins/harness-everything/skills/rewrite-commits/SKILL.md +37 -0
- package/plugins/harness-everything/skills/rewrite-commits/references/workflow.md +18 -0
- package/plugins/harness-everything/skills/security-review/SKILL.md +47 -0
- package/plugins/harness-everything/skills/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/plugins/harness-everything/skills/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/plugins/harness-everything/skills/security-review/references/security-checklist.md +478 -0
- package/plugins/harness-everything/skills/security-review/scripts/audit-secrets.js +85 -0
- package/plugins/harness-everything/skills/self-evolve/SKILL.md +45 -0
- package/plugins/harness-everything/skills/self-evolve/references/memory-resolution.md +61 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/persist-memory.js +272 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/register-dynamic-skill.js +253 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/self-regression.js +125 -0
- package/plugins/harness-everything/skills/skill-creator/SKILL.md +47 -0
- package/plugins/harness-everything/skills/skill-creator/references/authoring-workflow.md +14 -0
- package/plugins/harness-everything/skills/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-checklist.md +16 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-principles.md +81 -0
- package/plugins/harness-everything/skills/skill-creator/references/testing-workflow.md +27 -0
- package/plugins/harness-everything/skills/skill-style/SKILL.md +42 -0
- package/plugins/harness-everything/skills/skill-style/references/style-guide.md +62 -0
- package/plugins/harness-everything/skills/tdd/SKILL.md +38 -0
- package/plugins/harness-everything/skills/tdd/fixtures/pass-evidence.json +100 -0
- package/plugins/harness-everything/skills/tdd/fixtures/quality-scenarios.json +20 -0
- package/plugins/harness-everything/skills/tdd/guides/deep-modules.md +33 -0
- package/plugins/harness-everything/skills/tdd/guides/interface-design.md +31 -0
- package/plugins/harness-everything/skills/tdd/guides/mocking.md +59 -0
- package/plugins/harness-everything/skills/tdd/guides/refactoring.md +10 -0
- package/plugins/harness-everything/skills/tdd/guides/tests.md +61 -0
- package/plugins/harness-everything/skills/tdd/references/core-discipline.md +51 -0
- package/plugins/harness-everything/skills/tdd/references/integration-testing.md +13 -0
- package/plugins/harness-everything/skills/tdd/references/quality-model.md +68 -0
- package/plugins/harness-everything/skills/tdd/references/unit-testing.md +15 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-evidence.schema.json +211 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-report.schema.json +110 -0
- package/plugins/harness-everything/skills/tdd/scripts/quality-gate.js +475 -0
- package/plugins/harness-everything/skills/to-spec/SKILL.md +40 -0
- package/plugins/harness-everything/skills/to-spec/references/process.md +77 -0
- package/plugins/harness-everything/skills/to-spec/scripts/check-project-docs.js +239 -0
- package/plugins/harness-everything/skills/to-spec/scripts/project-docs-resolver.js +111 -0
- package/plugins/harness-everything/skills/to-spec/templates/cli-reference.md +31 -0
- package/plugins/harness-everything/skills/to-spec/templates/dev-doc.md +27 -0
- package/plugins/harness-everything/skills/to-spec/templates/feature-spec.md +59 -0
- package/plugins/harness-everything/skills/to-spec/templates/schema-doc.md +31 -0
- package/plugins/harness-everything/skills/to-tickets/SKILL.md +42 -0
- package/plugins/harness-everything/skills/to-tickets/references/publishing-and-templates.md +97 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/SKILL.md +42 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/references/execution-guide.md +27 -0
- package/plugins/harness-everything/skills/using-git-worktrees/SKILL.md +49 -0
- package/plugins/harness-everything/skills/using-git-worktrees/references/workflow-details.md +108 -0
- package/plugins/harness-everything/skills/verification-loop/SKILL.md +46 -0
- package/plugins/harness-everything/skills/verification-loop/references/verification-phases.md +116 -0
- package/plugins/harness-everything/skills/verification-loop/templates/verification-report.template.md +15 -0
- package/plugins/harness-everything/skills/verify-before-claim/SKILL.md +43 -0
- package/plugins/harness-everything/skills/verify-before-claim/references/verification-guide.md +73 -0
- package/plugins/harness-everything/skills/zoom-out/SKILL.md +42 -0
- package/plugins/harness-everything/skills/zoom-out/references/circuit-breaker.md +66 -0
- package/plugins/harness-everything/skills/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/repo-docs/SKILL.md +3 -7
- package/rewrite-commits/SKILL.md +2 -2
- package/scripts/build-openai-submission.js +158 -0
- package/scripts/installer.js +86 -11
- package/scripts/lib/advisory-text.js +25 -18
- package/scripts/lib/execution-contract.js +70 -0
- package/scripts/lib/manifest.js +34 -38
- package/scripts/lib/path-boundary.js +90 -0
- package/scripts/lib/skills.js +124 -70
- package/scripts/lib/workspace.js +269 -7
- package/scripts/sync-openai-plugin.js +91 -0
- package/scripts/sync-release-version.js +212 -0
- package/security-review/SKILL.md +4 -7
- package/security-review/scripts/audit-secrets.js +9 -8
- package/self-evolve/SKILL.md +9 -11
- package/self-evolve/references/memory-resolution.md +4 -1
- package/self-evolve/scripts/persist-memory.js +221 -88
- package/self-evolve/scripts/register-dynamic-skill.js +12 -10
- package/self-evolve/scripts/self-regression.js +23 -8
- package/skill-creator/SKILL.md +12 -14
- package/skill-creator/references/quality-checklist.md +1 -0
- package/skill-style/SKILL.md +4 -4
- package/skill-style/references/style-guide.md +29 -3
- package/submission/openai/README.md +67 -0
- package/submission/openai/listing.json +31 -0
- package/submission/openai/test-cases.json +61 -0
- package/tdd/SKILL.md +20 -26
- package/tdd/fixtures/pass-evidence.json +100 -0
- package/tdd/fixtures/quality-scenarios.json +20 -0
- package/tdd/references/core-discipline.md +2 -0
- package/tdd/references/integration-testing.md +13 -0
- package/tdd/references/quality-model.md +68 -0
- package/tdd/references/unit-testing.md +15 -0
- package/tdd/schemas/quality-evidence.schema.json +211 -0
- package/tdd/schemas/quality-report.schema.json +110 -0
- package/tdd/scripts/quality-gate.js +475 -0
- package/to-spec/SKILL.md +3 -5
- package/to-spec/scripts/check-project-docs.js +42 -62
- package/to-spec/scripts/project-docs-resolver.js +111 -0
- package/to-tickets/SKILL.md +4 -4
- package/todo-driven-workflow/SKILL.md +6 -4
- package/using-git-worktrees/SKILL.md +3 -3
- package/verification-loop/SKILL.md +4 -4
- package/verify-before-claim/SKILL.md +2 -2
- package/zoom-out/SKILL.md +4 -4
- package/ab-test-deep-analysis-2026-08-27.md +0 -383
- package/ab-test-results-2026-08-27.md +0 -152
- package/ab-test-statistical-analysis-31cases.md +0 -193
- package/opencode-plugin/hooks/circuit-breaker.js +0 -121
- package/opencode-plugin/hooks/compliance.js +0 -128
- package/opencode-plugin/hooks/post-edit.js +0 -62
- package/opencode-plugin/hooks/pre-complete.js +0 -65
- package/opencode-plugin/test-hook-firing.js +0 -198
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
const fs = require('fs');
|
|
5
|
+
const os = require('os');
|
|
6
|
+
const path = require('path');
|
|
7
|
+
const { spawnSync } = require('child_process');
|
|
8
|
+
|
|
9
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
10
|
+
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'harness-phase3-edge-'));
|
|
11
|
+
const workspace = path.join(tempRoot, 'workspace');
|
|
12
|
+
const stateHome = path.join(tempRoot, 'state-home');
|
|
13
|
+
fs.mkdirSync(path.join(workspace, '.git'), { recursive: true });
|
|
14
|
+
process.env.HARNESS_STATE_HOME = stateHome;
|
|
15
|
+
|
|
16
|
+
const { buildRouterContract } = require(path.join(ROOT, 'harness-everything', 'scripts', 'router-contract.js'));
|
|
17
|
+
const { prepareRun } = require(path.join(ROOT, 'fable-mode', 'scripts', 'workflow-plan-consumer.js'));
|
|
18
|
+
const { pathWithinScope } = require(path.join(ROOT, 'hooks', 'scripts', 'lib', 'fable-contracts.js'));
|
|
19
|
+
|
|
20
|
+
let failed = 0;
|
|
21
|
+
function check(condition, message) {
|
|
22
|
+
if (condition) console.log(` PASS ${message}`);
|
|
23
|
+
else { console.error(` FAIL ${message}`); failed++; }
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
const staged = buildRouterContract({
|
|
27
|
+
routingStatus: 'ok',
|
|
28
|
+
recommendedTier: 'Tier 3 (Macro Task)',
|
|
29
|
+
rationale: 'phase3 edge fixture',
|
|
30
|
+
reasonCodes: ['phase3-edge'],
|
|
31
|
+
signals: { macroScope: true, dependentStages: true },
|
|
32
|
+
});
|
|
33
|
+
|
|
34
|
+
const overlapping = prepareRun({
|
|
35
|
+
routerContract: staged,
|
|
36
|
+
workspaceRoot: workspace,
|
|
37
|
+
runId: 'staged-overlap',
|
|
38
|
+
stages: [
|
|
39
|
+
{ stageId: 'a', goal: 'a', agent: 'worker', task: 'a', dependsOn: [], writeSet: ['src/shared'], inputs: [], expectedOutputs: [] },
|
|
40
|
+
{ stageId: 'b', goal: 'b', agent: 'worker', task: 'b', dependsOn: [], writeSet: ['src/shared/file.js'], inputs: [], expectedOutputs: [] },
|
|
41
|
+
],
|
|
42
|
+
});
|
|
43
|
+
check(JSON.stringify(overlapping.execution.batches) === JSON.stringify([['a'], ['b']]), 'fable-staged serializes overlapping writers instead of rejecting them');
|
|
44
|
+
check(pathWithinScope('src/auth/token.js', 'src/auth'), 'scope matcher accepts already-normalized paths');
|
|
45
|
+
|
|
46
|
+
const isolated = prepareRun({
|
|
47
|
+
routerContract: staged,
|
|
48
|
+
workspaceRoot: workspace,
|
|
49
|
+
runId: 'session-b-only',
|
|
50
|
+
sessionId: 'session-b',
|
|
51
|
+
stages: [{
|
|
52
|
+
stageId: 'verify', goal: 'verify', agent: 'fable-verifier', task: 'verify', dependsOn: [], writeSet: [],
|
|
53
|
+
inputs: [], expectedOutputs: [], checkCommand: 'node --test isolated.test.js', passCondition: 'exit 0',
|
|
54
|
+
}],
|
|
55
|
+
});
|
|
56
|
+
const hook = path.join(ROOT, 'hooks', 'scripts', 'contract-test.js');
|
|
57
|
+
const result = spawnSync(process.execPath, [hook], {
|
|
58
|
+
cwd: workspace,
|
|
59
|
+
input: JSON.stringify({
|
|
60
|
+
hook_event_name: 'PostToolUse', tool_name: 'Bash', session_id: 'session-a', cwd: workspace,
|
|
61
|
+
tool_input: { command: 'node --test isolated.test.js' }, tool_response: { exitCode: 0, stdout: 'PASS' },
|
|
62
|
+
}),
|
|
63
|
+
encoding: 'utf8',
|
|
64
|
+
env: { ...process.env, HARNESS_STATE_HOME: stateHome },
|
|
65
|
+
});
|
|
66
|
+
check(result.status === 2, 'a check from another session cannot resolve this run contract');
|
|
67
|
+
const manifest = JSON.parse(fs.readFileSync(path.join(isolated.runRoot, 'contracts', 'verify.json'), 'utf8'));
|
|
68
|
+
check(manifest.status === 'planned', 'cross-session check leaves the target contract unmodified');
|
|
69
|
+
|
|
70
|
+
try { fs.rmSync(tempRoot, { recursive: true, force: true }); } catch (_) {}
|
|
71
|
+
console.log(`\n${failed === 0 ? 'PASS' : 'FAIL'}: #85 Phase 3 edge contracts (${failed} failure${failed === 1 ? '' : 's'})`);
|
|
72
|
+
process.exit(failed === 0 ? 0 : 1);
|
|
@@ -0,0 +1,276 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict';
|
|
3
|
+
|
|
4
|
+
const assert = require('assert');
|
|
5
|
+
const fs = require('fs');
|
|
6
|
+
const os = require('os');
|
|
7
|
+
const path = require('path');
|
|
8
|
+
const { spawnSync } = require('child_process');
|
|
9
|
+
|
|
10
|
+
const ROOT = path.resolve(__dirname, '..');
|
|
11
|
+
const tempRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'harness-phase3-'));
|
|
12
|
+
const stateHome = path.join(tempRoot, 'state-home');
|
|
13
|
+
const workspace = path.join(tempRoot, 'workspace');
|
|
14
|
+
fs.mkdirSync(workspace, { recursive: true });
|
|
15
|
+
fs.mkdirSync(path.join(workspace, '.git'), { recursive: true });
|
|
16
|
+
process.env.HARNESS_STATE_HOME = stateHome;
|
|
17
|
+
|
|
18
|
+
const { buildRouterContract } = require(path.join(ROOT, 'harness-everything', 'scripts', 'router-contract.js'));
|
|
19
|
+
const {
|
|
20
|
+
derivePlanId,
|
|
21
|
+
getWorkspaceStateRoot,
|
|
22
|
+
prepareRun,
|
|
23
|
+
validateStageGraph,
|
|
24
|
+
} = require(path.join(ROOT, 'fable-mode', 'scripts', 'workflow-plan-consumer.js'));
|
|
25
|
+
const { consumeWorkspacePlan } = require(path.join(ROOT, 'multi-agent-workspace', 'scripts', 'consume-workflow-plan.js'));
|
|
26
|
+
const { pathWithinScope } = require(path.join(ROOT, 'hooks', 'scripts', 'lib', 'fable-contracts.js'));
|
|
27
|
+
|
|
28
|
+
let failed = 0;
|
|
29
|
+
function check(condition, message) {
|
|
30
|
+
if (condition) console.log(` PASS ${message}`);
|
|
31
|
+
else {
|
|
32
|
+
console.error(` FAIL ${message}`);
|
|
33
|
+
failed++;
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
function expectThrow(fn, pattern, message) {
|
|
38
|
+
try {
|
|
39
|
+
fn();
|
|
40
|
+
check(false, message);
|
|
41
|
+
} catch (err) {
|
|
42
|
+
check(pattern.test(err.message), `${message} (${err.message})`);
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
function contractFor(signals) {
|
|
47
|
+
return buildRouterContract({
|
|
48
|
+
routingStatus: 'ok',
|
|
49
|
+
recommendedTier: 'Tier 3 (Macro Task)',
|
|
50
|
+
rationale: 'phase 3 test fixture',
|
|
51
|
+
reasonCodes: ['phase3-test'],
|
|
52
|
+
signals: { macroScope: true, ...signals },
|
|
53
|
+
});
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
const parallelContract = contractFor({ independentWorkstreams: true, readOnly: true });
|
|
57
|
+
const stagedContract = contractFor({ dependentStages: true });
|
|
58
|
+
const workspaceContract = contractFor({ multiSession: true, reusableSpecialists: true, crossDomain: true });
|
|
59
|
+
|
|
60
|
+
console.log('=== #85 Phase 3 Workflow Plan Consumers ===');
|
|
61
|
+
check(parallelContract.workflowPlan.strategy === 'fable-parallel', 'fixture selects fable-parallel');
|
|
62
|
+
check(stagedContract.workflowPlan.strategy === 'fable-staged', 'fixture selects fable-staged');
|
|
63
|
+
check(workspaceContract.workflowPlan.strategy === 'fable-multi-agent-workspace', 'fixture selects fable-multi-agent-workspace');
|
|
64
|
+
|
|
65
|
+
const planIdA = derivePlanId(parallelContract);
|
|
66
|
+
const planIdB = derivePlanId(JSON.parse(JSON.stringify(parallelContract)));
|
|
67
|
+
check(planIdA === planIdB && /^plan-[0-9a-f]{16}$/.test(planIdA), 'planId is deterministic and content-addressed');
|
|
68
|
+
|
|
69
|
+
const parallelStages = [
|
|
70
|
+
{
|
|
71
|
+
stageId: 'security', goal: 'read security state', agent: 'fable-worker-sonnet', task: 'audit security',
|
|
72
|
+
inputs: [], expectedOutputs: ['security notes'], outputPath: null, dependsOn: [], writeSet: [],
|
|
73
|
+
checkCommand: 'node --test security.test.js', passCondition: 'exit 0',
|
|
74
|
+
},
|
|
75
|
+
{
|
|
76
|
+
stageId: 'architecture', goal: 'read architecture state', agent: 'fable-worker-sonnet', task: 'audit architecture',
|
|
77
|
+
inputs: [], expectedOutputs: ['architecture notes'], outputPath: null, dependsOn: [], writeSet: [],
|
|
78
|
+
checkCommand: 'node --test architecture.test.js', passCondition: 'exit 0',
|
|
79
|
+
},
|
|
80
|
+
{
|
|
81
|
+
stageId: 'synthesis', goal: 'synthesize results', agent: 'fable-worker-sonnet', task: 'write synthesis',
|
|
82
|
+
inputs: [], expectedOutputs: ['summary'], outputPath: 'reports/summary.md', dependsOn: ['security', 'architecture'],
|
|
83
|
+
writeSet: ['reports/summary.md'], checkCommand: 'node --test synthesis.test.js', passCondition: 'exit 0',
|
|
84
|
+
},
|
|
85
|
+
];
|
|
86
|
+
|
|
87
|
+
const runA = prepareRun({
|
|
88
|
+
routerContract: parallelContract,
|
|
89
|
+
stages: parallelStages,
|
|
90
|
+
workspaceRoot: workspace,
|
|
91
|
+
runId: 'phase3-run-a',
|
|
92
|
+
sessionId: 'phase3-session',
|
|
93
|
+
});
|
|
94
|
+
check(JSON.stringify(runA.execution.batches) === JSON.stringify([['architecture', 'security'], ['synthesis']]), 'parallel plan emits dependency-safe ready batches');
|
|
95
|
+
check(fs.existsSync(path.join(runA.runRoot, 'contracts', 'security.json')), 'run-scoped stage contract is written');
|
|
96
|
+
check(fs.existsSync(path.join(runA.runRoot, 'contracts', 'synthesis.json')), 'downstream stage has its own contract');
|
|
97
|
+
|
|
98
|
+
const runB = prepareRun({
|
|
99
|
+
routerContract: parallelContract,
|
|
100
|
+
stages: parallelStages,
|
|
101
|
+
workspaceRoot: workspace,
|
|
102
|
+
runId: 'phase3-run-b',
|
|
103
|
+
sessionId: 'other-session',
|
|
104
|
+
});
|
|
105
|
+
check(runA.planId === runB.planId, 'same router plan shares deterministic planId across executions');
|
|
106
|
+
check(runA.runRoot !== runB.runRoot, 'same stageIds in concurrent runs never share mutable contract paths');
|
|
107
|
+
|
|
108
|
+
const stagedRun = prepareRun({
|
|
109
|
+
routerContract: stagedContract,
|
|
110
|
+
stages: [
|
|
111
|
+
{ stageId: 'one', goal: 'one', agent: 'fable-worker-sonnet', task: 'one', dependsOn: [], writeSet: ['src/one'], inputs: [], expectedOutputs: [] },
|
|
112
|
+
{ stageId: 'two', goal: 'two', agent: 'fable-worker-sonnet', task: 'two', dependsOn: [], writeSet: ['src/two'], inputs: [], expectedOutputs: [] },
|
|
113
|
+
],
|
|
114
|
+
workspaceRoot: workspace,
|
|
115
|
+
runId: 'phase3-staged',
|
|
116
|
+
});
|
|
117
|
+
check(JSON.stringify(stagedRun.execution.batches) === JSON.stringify([['one'], ['two']]), 'fable-staged serializes otherwise independent stages');
|
|
118
|
+
|
|
119
|
+
expectThrow(() => validateStageGraph([
|
|
120
|
+
{ stageId: 'broken', goal: 'x', agent: 'a', task: 'x', dependsOn: ['missing'], writeSet: [], inputs: [], expectedOutputs: [] },
|
|
121
|
+
]), /unresolved dependency/, 'unknown dependency is rejected');
|
|
122
|
+
|
|
123
|
+
expectThrow(() => validateStageGraph([
|
|
124
|
+
{ stageId: 'a', goal: 'a', agent: 'a', task: 'a', dependsOn: ['b'], writeSet: [], inputs: [], expectedOutputs: [] },
|
|
125
|
+
{ stageId: 'b', goal: 'b', agent: 'a', task: 'b', dependsOn: ['a'], writeSet: [], inputs: [], expectedOutputs: [] },
|
|
126
|
+
]), /cyclic/, 'dependency cycle is rejected');
|
|
127
|
+
|
|
128
|
+
expectThrow(() => prepareRun({
|
|
129
|
+
routerContract: parallelContract,
|
|
130
|
+
workspaceRoot: workspace,
|
|
131
|
+
runId: 'phase3-overlap',
|
|
132
|
+
stages: [
|
|
133
|
+
{ stageId: 'a', goal: 'a', agent: 'a', task: 'a', dependsOn: [], writeSet: ['src/auth'], inputs: [], expectedOutputs: [] },
|
|
134
|
+
{ stageId: 'b', goal: 'b', agent: 'a', task: 'b', dependsOn: [], writeSet: ['src/auth/token.js'], inputs: [], expectedOutputs: [] },
|
|
135
|
+
],
|
|
136
|
+
}), /write-set overlap/, 'parallel write-set overlap rejects dispatch');
|
|
137
|
+
|
|
138
|
+
check(pathWithinScope(' M src/auth/token.js', 'src/auth'), 'scope matcher accepts descendants of a declared path');
|
|
139
|
+
check(!pathWithinScope(' M src/author.js', 'src/auth'), 'scope matcher uses path boundaries, not string prefixes');
|
|
140
|
+
|
|
141
|
+
// Workspace consumer: preserve existing role/memory ownership and add only correlation.
|
|
142
|
+
const aiDir = path.join(workspace, '.ai');
|
|
143
|
+
fs.mkdirSync(aiDir, { recursive: true });
|
|
144
|
+
const originalHandoff = {
|
|
145
|
+
schemaVersion: 1,
|
|
146
|
+
state: 'ready',
|
|
147
|
+
workspaceKey: 'fixture',
|
|
148
|
+
repoRoot: workspace,
|
|
149
|
+
selectedAgents: [{ id: 'security-reviewer', provider: 'agency-agents', source: 'catalog/security.md' }],
|
|
150
|
+
memoryIndex: '.ai/memory/index.json',
|
|
151
|
+
generatedAt: 'fixture',
|
|
152
|
+
};
|
|
153
|
+
fs.writeFileSync(path.join(aiDir, 'handoff.json'), JSON.stringify(originalHandoff, null, 2));
|
|
154
|
+
const correlatedHandoff = consumeWorkspacePlan({
|
|
155
|
+
routerContract: workspaceContract,
|
|
156
|
+
workspaceRoot: workspace,
|
|
157
|
+
runId: 'phase3-workspace',
|
|
158
|
+
});
|
|
159
|
+
check(JSON.stringify(correlatedHandoff.selectedAgents) === JSON.stringify(originalHandoff.selectedAgents), 'workspace consumer preserves selected roles/provenance');
|
|
160
|
+
check(correlatedHandoff.memoryIndex === originalHandoff.memoryIndex, 'workspace consumer preserves existing memory index');
|
|
161
|
+
check(correlatedHandoff.workflowCorrelation.planId === derivePlanId(workspaceContract), 'workspace handoff is correlated to router planId');
|
|
162
|
+
check(correlatedHandoff.workflowCorrelation.runId === 'phase3-workspace', 'workspace handoff records shared runId');
|
|
163
|
+
check(correlatedHandoff.workflowCorrelation.rolePolicy === 'existing-workspace-selection', 'router does not take ownership of role selection');
|
|
164
|
+
|
|
165
|
+
// Exact stage-check correlation and evidence.
|
|
166
|
+
const checkRun = prepareRun({
|
|
167
|
+
routerContract: stagedContract,
|
|
168
|
+
workspaceRoot: workspace,
|
|
169
|
+
runId: 'phase3-check',
|
|
170
|
+
sessionId: 'check-session',
|
|
171
|
+
stages: [{
|
|
172
|
+
stageId: 'verify', goal: 'verify', agent: 'fable-verifier', task: 'verify output', dependsOn: [], writeSet: [],
|
|
173
|
+
inputs: [], expectedOutputs: ['evidence'], checkCommand: 'node --test phase3.test.js', passCondition: 'exit 0',
|
|
174
|
+
}],
|
|
175
|
+
});
|
|
176
|
+
const contractHook = path.join(ROOT, 'hooks', 'scripts', 'contract-test.js');
|
|
177
|
+
const checkPayload = {
|
|
178
|
+
hook_event_name: 'PostToolUse',
|
|
179
|
+
tool_name: 'Bash',
|
|
180
|
+
session_id: 'check-session',
|
|
181
|
+
cwd: workspace,
|
|
182
|
+
tool_input: { command: 'node --test phase3.test.js' },
|
|
183
|
+
tool_response: { exitCode: 0, stdout: 'PASS phase3' },
|
|
184
|
+
};
|
|
185
|
+
const checkResult = spawnSync(process.execPath, [contractHook], {
|
|
186
|
+
cwd: workspace,
|
|
187
|
+
input: JSON.stringify(checkPayload),
|
|
188
|
+
encoding: 'utf8',
|
|
189
|
+
env: { ...process.env, HARNESS_STATE_HOME: stateHome },
|
|
190
|
+
});
|
|
191
|
+
check(checkResult.status === 0, `exact correlated stage check passes (${checkResult.stderr.trim() || 'no stderr'})`);
|
|
192
|
+
const resolvedContract = JSON.parse(fs.readFileSync(path.join(checkRun.runRoot, 'contracts', 'verify.json'), 'utf8'));
|
|
193
|
+
check(resolvedContract.status === 'pass', 'contract-test resolves exactly one run-scoped stage contract');
|
|
194
|
+
check(resolvedContract.verificationEvidence === 'evidence/verify.json', 'stage contract links run-scoped verification evidence');
|
|
195
|
+
const evidence = JSON.parse(fs.readFileSync(path.join(checkRun.runRoot, 'evidence', 'verify.json'), 'utf8'));
|
|
196
|
+
check(evidence.planId === checkRun.planId && evidence.runId === checkRun.runId && evidence.stageId === 'verify', 'verification evidence correlates plan/run/stage');
|
|
197
|
+
|
|
198
|
+
// Same command in two active contracts must not mutate either without enough correlation.
|
|
199
|
+
for (const runId of ['ambiguous-a', 'ambiguous-b']) {
|
|
200
|
+
prepareRun({
|
|
201
|
+
routerContract: stagedContract,
|
|
202
|
+
workspaceRoot: workspace,
|
|
203
|
+
runId,
|
|
204
|
+
sessionId: 'ambiguous-session',
|
|
205
|
+
stages: [{
|
|
206
|
+
stageId: 'same-check', goal: 'verify', agent: 'fable-verifier', task: 'verify', dependsOn: [], writeSet: [],
|
|
207
|
+
inputs: [], expectedOutputs: [], checkCommand: 'npm test', passCondition: 'exit 0',
|
|
208
|
+
}],
|
|
209
|
+
});
|
|
210
|
+
}
|
|
211
|
+
const ambiguousResult = spawnSync(process.execPath, [contractHook], {
|
|
212
|
+
cwd: workspace,
|
|
213
|
+
input: JSON.stringify({
|
|
214
|
+
hook_event_name: 'PostToolUse', tool_name: 'Bash', session_id: 'ambiguous-session', cwd: workspace,
|
|
215
|
+
tool_input: { command: 'npm test' }, tool_response: { exitCode: 0, stdout: 'ok' },
|
|
216
|
+
}),
|
|
217
|
+
encoding: 'utf8',
|
|
218
|
+
env: { ...process.env, HARNESS_STATE_HOME: stateHome },
|
|
219
|
+
});
|
|
220
|
+
check(ambiguousResult.status === 2, 'ambiguous same-command correlation blocks acceptance');
|
|
221
|
+
for (const runId of ['ambiguous-a', 'ambiguous-b']) {
|
|
222
|
+
const manifest = JSON.parse(fs.readFileSync(path.join(getWorkspaceStateRoot(workspace), 'fable-runs', runId, 'contracts', 'same-check.json'), 'utf8'));
|
|
223
|
+
check(manifest.status === 'planned', `${runId} remains unmodified after ambiguous check`);
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
// Full subagent-scope hook: declared path passes; undeclared path fails.
|
|
227
|
+
const gitWorkspace = path.join(tempRoot, 'git-workspace');
|
|
228
|
+
fs.mkdirSync(path.join(gitWorkspace, 'src', 'auth'), { recursive: true });
|
|
229
|
+
fs.mkdirSync(path.join(gitWorkspace, 'docs'), { recursive: true });
|
|
230
|
+
fs.writeFileSync(path.join(gitWorkspace, 'src', 'auth', 'token.txt'), 'base\n');
|
|
231
|
+
fs.writeFileSync(path.join(gitWorkspace, 'docs', 'base.txt'), 'base\n');
|
|
232
|
+
for (const args of [
|
|
233
|
+
['init'],
|
|
234
|
+
['config', 'user.email', 'phase3@example.invalid'],
|
|
235
|
+
['config', 'user.name', 'Phase3 Test'],
|
|
236
|
+
['add', '.'],
|
|
237
|
+
['commit', '-m', 'baseline'],
|
|
238
|
+
]) {
|
|
239
|
+
const result = spawnSync('git', args, { cwd: gitWorkspace, encoding: 'utf8' });
|
|
240
|
+
assert.strictEqual(result.status, 0, result.stderr);
|
|
241
|
+
}
|
|
242
|
+
prepareRun({
|
|
243
|
+
routerContract: stagedContract,
|
|
244
|
+
workspaceRoot: gitWorkspace,
|
|
245
|
+
runId: 'scope-run',
|
|
246
|
+
sessionId: 'scope-session',
|
|
247
|
+
stages: [{ stageId: 'edit-auth', goal: 'edit auth', agent: 'worker', task: 'edit auth', dependsOn: [], writeSet: ['src/auth'], inputs: [], expectedOutputs: [] }],
|
|
248
|
+
});
|
|
249
|
+
const scopeHook = path.join(ROOT, 'hooks', 'scripts', 'subagent-scope-guard.js');
|
|
250
|
+
function runScope(event) {
|
|
251
|
+
return spawnSync(process.execPath, [scopeHook], {
|
|
252
|
+
cwd: gitWorkspace,
|
|
253
|
+
input: JSON.stringify({ hook_event_name: event, tool_name: 'Task', session_id: 'scope-session', cwd: gitWorkspace }),
|
|
254
|
+
encoding: 'utf8',
|
|
255
|
+
env: { ...process.env, HARNESS_STATE_HOME: stateHome },
|
|
256
|
+
});
|
|
257
|
+
}
|
|
258
|
+
check(runScope('PreToolUse').status === 0, 'scope guard captures machine-readable write-set snapshot');
|
|
259
|
+
fs.writeFileSync(path.join(gitWorkspace, 'src', 'auth', 'token.txt'), 'changed\n');
|
|
260
|
+
const inScope = runScope('PostToolUse');
|
|
261
|
+
check(inScope.status === 0 && /in-scope/.test(inScope.stdout), 'declared in-scope worker edit is attributed and accepted');
|
|
262
|
+
check(runScope('PreToolUse').status === 0, 'next burst starts from rolled-forward baseline');
|
|
263
|
+
fs.writeFileSync(path.join(gitWorkspace, 'docs', 'oops.txt'), 'unexpected\n');
|
|
264
|
+
const outOfScope = runScope('PostToolUse');
|
|
265
|
+
check(outOfScope.status === 2 && /OUT-OF-SCOPE/.test(outOfScope.stderr), 'undeclared worker edit is rejected by scope guard');
|
|
266
|
+
|
|
267
|
+
const contractDoc = fs.readFileSync(path.join(ROOT, 'fable-mode', 'CONTRACT-FORMAT.md'), 'utf8');
|
|
268
|
+
const orchestratorDoc = fs.readFileSync(path.join(ROOT, 'fable-mode', 'agents', 'fable-orchestrator.md'), 'utf8');
|
|
269
|
+
const workspaceDoc = fs.readFileSync(path.join(ROOT, 'multi-agent-workspace', 'references', 'orchestration.md'), 'utf8');
|
|
270
|
+
check(contractDoc.includes('dependsOn') && contractDoc.includes('writeSet'), 'Fable contract documentation requires dependency/write-set declarations');
|
|
271
|
+
check(orchestratorDoc.includes('workflow-plan-consumer.js') && orchestratorDoc.includes('planId'), 'Fable orchestrator consumes structured plan and correlation ids');
|
|
272
|
+
check(workspaceDoc.includes('does not decide whether a task deserves multiple agents'), 'workspace no longer independently chooses topology from task size');
|
|
273
|
+
|
|
274
|
+
try { fs.rmSync(tempRoot, { recursive: true, force: true }); } catch (_) {}
|
|
275
|
+
console.log(`\n${failed === 0 ? 'PASS' : 'FAIL'}: #85 Phase 3 workflow-plan consumers (${failed} failure${failed === 1 ? '' : 's'})`);
|
|
276
|
+
process.exit(failed === 0 ? 0 : 1);
|
package/docs/architecture.md
CHANGED
|
@@ -1,99 +1,179 @@
|
|
|
1
1
|
# Harness Architecture
|
|
2
2
|
|
|
3
|
-
This document
|
|
3
|
+
This document describes the internal architecture, lifecycle, and integration boundaries of the Harness behavior layer. The current cross-platform claim boundary is centralized in [platform-capabilities.md](platform-capabilities.md); host sections here must remain consistent with it.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
7
|
## Architectural Overview
|
|
8
8
|
|
|
9
|
-
Harness
|
|
9
|
+
Harness is a **system supervisor**, not a universal workflow engine. Skills remain independently useful, while runtime mechanisms establish a small cross-cutting contract around software work.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
The architecture follows a mechanism-first skill mesh with a **minimal kernel**:
|
|
12
12
|
|
|
13
|
-
|
|
13
|
+
- classify scope before mutation,
|
|
14
|
+
- require evidence before completion claims,
|
|
15
|
+
- stop repeated same-signature micro-retries and re-plan,
|
|
16
|
+
- leave domain skill selection, ordering, planning style, and delegation to the agent.
|
|
17
|
+
|
|
18
|
+
> **Do not enforce workflow order. Enforce workflow invariants.**
|
|
19
|
+
|
|
20
|
+
This is intentionally different from the older design where Tier 2 implied a fixed TDD/checklist pipeline and Tier 3 implied a mandatory Fable/multi-agent pipeline.
|
|
14
21
|
|
|
15
22
|
```mermaid
|
|
16
23
|
flowchart TD
|
|
17
24
|
subgraph User_Session [User Session]
|
|
18
|
-
U([User Request]) --> Boot[bootstrap.js: Session Start]
|
|
25
|
+
U([User Request]) --> Boot[bootstrap.js: Session Start / restore]
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
subgraph Kernel [Harness Kernel]
|
|
29
|
+
Boot --> Preflight[Environment / integration discovery]
|
|
30
|
+
Preflight --> KR[kernel-router.js]
|
|
31
|
+
KR --> TR[tier-router.js<br/>classifier + guide discovery]
|
|
32
|
+
TR --> Inv[Inject minimal invariants]
|
|
33
|
+
Inv --> Choice[Agent chooses useful skills / tactics]
|
|
19
34
|
end
|
|
20
35
|
|
|
21
|
-
subgraph
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
Router -->|Tier 2: Standard Task| T2[tdd: Test-Driven Development]
|
|
27
|
-
Router -->|Tier 3: Macro Task| T3[fable-mode: Multi-Agent Spawn]
|
|
28
|
-
T2 & T3 --> TDW[todo-driven-workflow: Base Execution Checklist]
|
|
36
|
+
subgraph Execution [Agent-Controlled Execution]
|
|
37
|
+
Choice --> Tools[Tool calls / edits / tests]
|
|
38
|
+
Tools --> Evidence{Evidence supports completion?}
|
|
39
|
+
Evidence -- Yes --> Done[Evidence-backed completion]
|
|
40
|
+
Evidence -- No --> Retry[Diagnose / iterate]
|
|
29
41
|
end
|
|
30
42
|
|
|
31
|
-
subgraph
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
CB -->|Fails 3x| ZO(zoom-out: Halt, Reflect & Fact-Check)
|
|
37
|
-
ZO -->|Fresh diagnosis: RESUME| Tools
|
|
38
|
-
CB -->|Succeeds| Done[Success / Finish]
|
|
43
|
+
subgraph Defense [Runtime Boundaries]
|
|
44
|
+
Retry --> CB{Same-signature failure x3?}
|
|
45
|
+
CB -- No --> Tools
|
|
46
|
+
CB -- Yes --> ZO[zoom-out / fresh diagnosis]
|
|
47
|
+
ZO --> Tools
|
|
39
48
|
end
|
|
40
49
|
|
|
41
|
-
subgraph
|
|
42
|
-
|
|
43
|
-
Human --> SE[self-evolve: Deep Reflection]
|
|
44
|
-
Done -->|Complex Breakthrough| SE
|
|
45
|
-
SE --> RunSR[self-regression.js: CI Check]
|
|
46
|
-
RunSR -->|Pass 100%| Mem[(Long-term Memory / RULES.md)]
|
|
47
|
-
Mem -.->|Immunize| Preflight
|
|
50
|
+
subgraph Learning [Optional Learning / State]
|
|
51
|
+
Done --> Record[Record / self-evolve when useful]
|
|
48
52
|
end
|
|
49
53
|
|
|
50
|
-
style
|
|
51
|
-
style
|
|
52
|
-
style
|
|
54
|
+
style Kernel fill:#eceff1,stroke:#37474f,stroke-width:2px,color:#000000
|
|
55
|
+
style Defense fill:#fff9c4,stroke:#fbc02d,stroke-width:2px,color:#000000
|
|
56
|
+
style Learning fill:#e3f2fd,stroke:#1e88e5,stroke-width:2px,color:#000000
|
|
53
57
|
```
|
|
54
58
|
|
|
59
|
+
### Router responsibilities
|
|
60
|
+
|
|
61
|
+
`kernel-router.js` is the public runtime entry point. It delegates heuristic classification and dynamic guide/skill discovery to `tier-router.js`, then converts the result into the Harness contract:
|
|
62
|
+
|
|
63
|
+
- recommended Tier + rationale,
|
|
64
|
+
- required invariants,
|
|
65
|
+
- advisory skill suggestions.
|
|
66
|
+
|
|
67
|
+
The kernel deliberately suppresses the old fixed `BASE EXECUTION LOOP` wording so that a host selecting `tdd`, `security-review`, `repo-docs`, or another peer skill cannot accidentally replace or bypass the cross-cutting Harness contract.
|
|
68
|
+
|
|
69
|
+
`harness-everything` remains the public/manual skill entry point for routing, debugging, and re-routing. `install-cognitive-os` remains the explanatory/manual entry point for the Discover → Think → Try → Summarize → Record policy. Automatic correctness must not depend on either one winning host peer-skill selection first.
|
|
70
|
+
|
|
55
71
|
---
|
|
56
72
|
|
|
57
|
-
## Integration
|
|
73
|
+
## Integration Touchpoints
|
|
74
|
+
|
|
75
|
+
Harness aligns to each host's real capabilities. Enforcement strength is platform-specific; shared skill text does not prove mechanism parity.
|
|
76
|
+
|
|
77
|
+
### Self-healing and placement
|
|
78
|
+
|
|
79
|
+
`harness-everything/scripts/self-heal.js` audits supported integration touchpoints and can re-run the idempotent installer to repair missing pieces. If a user intentionally removed an integration, respect that choice.
|
|
80
|
+
|
|
81
|
+
A platform's native files stay where that platform expects them. Harness-owned runtime state converges under the platform's `harness-everything/` state directory, while skills remain in each platform's native skill location. `manifest.json` records installed artifacts so uninstall can remove only Harness-owned files.
|
|
82
|
+
|
|
83
|
+
### Runtime state vs. skill content
|
|
84
|
+
|
|
85
|
+
Runtime state includes hook metadata, circuit-breaker counters, handoff/verification timestamps, and WAL-style session state. Skill content is separate and independently discoverable. This separation lets a domain skill remain useful even when a host does not support Harness runtime hooks.
|
|
86
|
+
|
|
87
|
+
---
|
|
88
|
+
|
|
89
|
+
## Host Adapters
|
|
90
|
+
|
|
91
|
+
### 1. Claude Code — hook-enforced
|
|
92
|
+
|
|
93
|
+
The installer configures native lifecycle hooks and project skills.
|
|
94
|
+
|
|
95
|
+
- `SessionStart`: `bootstrap.js` restores prior state and audits integrations.
|
|
96
|
+
- `UserPromptSubmit`: `kernel-router.js` establishes routing + invariants before peer/domain skill execution.
|
|
97
|
+
- `PreToolUse`: circuit breaker, boundary/depth/context guards, and subagent scope guards can block supported tool calls.
|
|
98
|
+
- `PostToolUse`: records outcomes/state and tracks repeated failures.
|
|
99
|
+
- `Stop`: `stop-gate.js` prevents an edit batch from being claimed complete without successful verification evidence.
|
|
58
100
|
|
|
59
|
-
|
|
101
|
+
The prompt hook is intentionally lightweight: it does **not** prescribe TODO/TDD/Fable order. It gives the model the rails and lets the model orchestrate itself.
|
|
60
102
|
|
|
61
|
-
|
|
103
|
+
### 2. OpenCode — plugin enforcement, live loading still unverified
|
|
62
104
|
|
|
63
|
-
|
|
105
|
+
`opencode-plugin/` maps supported enforcement behavior to OpenCode's real plugin API. Source-level/mechanism tests exist, but live plugin loading remains unverified; do not promote source/mechanism evidence into a live-host enforcement claim until a real OpenCode session artifact exists.
|
|
64
106
|
|
|
65
|
-
|
|
107
|
+
### 3. Cursor — skills plus advisory rules
|
|
66
108
|
|
|
67
|
-
|
|
109
|
+
The current installer uses `.cursor/skills/` plus `.cursorrules`. Cursor’s documented skill and plugin surfaces are separate from Harness runtime hooks; routing/verification/retry boundaries remain self-directed unless a compatible mechanism is independently verified.
|
|
68
110
|
|
|
69
|
-
###
|
|
70
|
-
Our installer configures native lifecycle hooks inside `.claude/settings.json`, and places project-level skills inside the native `.claude/skills/` directory where Claude Code natively auto-discovers and registers them.
|
|
71
|
-
* `SessionStart`: Runs `bootstrap.js` to restore previous handoffs, check environment variables, and initialize session state.
|
|
72
|
-
* `PreToolUse`: Triggers `rule-of-3.js`, `boundary-guard.js`, `depth-guard.js`, `context-compact.js`, and `subagent-scope-guard.js` to intercept tool invocations before they run, and can actually block one (`exit(2)`) — e.g. the Rule of 3 circuit breaker.
|
|
73
|
-
* `PostToolUse`: Records tool outcomes, updates the persistent transaction log (WAL) via `state-persist.js`, tracks repeat failures, and can only add advisory context back (the tool already ran; nothing here blocks it).
|
|
74
|
-
* `Stop`: Runs `stop-gate.js` — when a turn ends with uncommitted edits that were never followed by a successful verification command (test/build/lint), it bounces the stop back once per edit batch. This is the mechanical form of `verification-loop`'s pre-delivery gate; on every other platform that gate remains advisory prose.
|
|
111
|
+
### 4. GitHub Copilot agent surfaces — skills plus advisory instructions
|
|
75
112
|
|
|
76
|
-
|
|
77
|
-
The installer appends guidance to `.cursorrules`. Cursor has no hook/execution mechanism, so nothing in `.claude/harness-everything/state/` gets read or written by Cursor itself, and no tool call can be blocked — the model is simply asked (in the same file, every session) to self-regulate: discover the environment before acting, stop after 3 repeated failures instead of continuing to retry, and prefer small commits.
|
|
113
|
+
The current installer uses `.github/skills/` plus `.github/copilot-instructions.md`. GitHub’s Agent Skills paths are documented, but no Harness-specific plugin or live session is verified here.
|
|
78
114
|
|
|
79
|
-
###
|
|
80
|
-
Same mechanism and same limits as Cursor, via `.github/copilot-instructions.md`.
|
|
115
|
+
### 5. Codex — local OpenAI plugin plus advisory installer path
|
|
81
116
|
|
|
82
|
-
|
|
83
|
-
Same mechanism and same limits again, via `AGENTS.md` — Codex's actual custom-instruction file, read automatically at session start (`.codex/config.toml` controls CLI/sandbox behavior, not prompt content, and was never a valid target for this).
|
|
117
|
+
Codex has two distinct Harness integration surfaces and documentation must keep them separate:
|
|
84
118
|
|
|
85
|
-
|
|
86
|
-
|
|
119
|
+
- The general `--codex` installer path writes Codex-facing skills/instructions such as `AGENTS.md`; where the host only consumes those instructions, that path is advisory.
|
|
120
|
+
- The packaged local OpenAI plugin under `plugins/harness-everything/` includes `.codex-plugin/plugin.json`, all 26 canonical skills, and lifecycle hooks for `SessionStart`, `UserPromptSubmit`, supported local `PreToolUse` / `PostToolUse`, `SubagentStart` / `SubagentStop`, and `Stop`. Those hooks inject the compact session policy, run the invariant-first kernel, track supported edits and verification, and surface subagent scope changes.
|
|
87
121
|
|
|
88
|
-
|
|
89
|
-
|
|
122
|
+
The local plugin therefore has **mechanism-tested local enforcement** for the packaged event/tool mappings. This is package evidence, not a live-host claim; the host's hook review/trust flow and a fresh session are still required. Do not infer broader Claude parity for events or tools that are not listed in the package.
|
|
123
|
+
|
|
124
|
+
The public OpenAI **Skills-only** submission is narrower again: it ships reusable skills and referenced assets, not the local `.codex-plugin` lifecycle hooks. Public reviewer/listing claims must describe skill/workflow behavior rather than local hook enforcement. See [openai-plugin.md](openai-plugin.md).
|
|
125
|
+
|
|
126
|
+
### 6. Continue.dev — rules documented, skill discovery unknown
|
|
127
|
+
|
|
128
|
+
The installer writes `.continue/rules/harness.md` with the platform's native rules format and retains `.continue/skills/` as an adapter candidate. The reviewed official Continue documentation does not establish standalone `SKILL.md` discovery, so no skill-loading or hard-gate parity is claimed.
|
|
129
|
+
|
|
130
|
+
### 7. Hermes Agent — trusted skills plus advisory context
|
|
131
|
+
|
|
132
|
+
The installer writes `.hermes.md` for explicit project coverage and can install trusted project skills under `.agents/skills/` plus global skills under `~/.hermes/skills/`. Project skill loading remains subject to Hermes trust; no hard-gate parity is claimed.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## Cognitive OS and Skill Mesh
|
|
137
|
+
|
|
138
|
+
The Cognitive OS is a policy layer, not a parent skill that every domain skill must call.
|
|
139
|
+
|
|
140
|
+
```mermaid
|
|
141
|
+
flowchart LR
|
|
142
|
+
D[Discover] --> T[Think]
|
|
143
|
+
T --> Y[Try]
|
|
144
|
+
Y --> S[Summarize]
|
|
145
|
+
S --> R[Record]
|
|
146
|
+
S -- insufficient evidence --> T
|
|
147
|
+
Y -- same failure x3 --> Z[Zoom Out]
|
|
148
|
+
Z --> T
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
A domain skill may have its own lifecycle — for example RED/GREEN/REFACTOR inside `tdd` — without being forced into a global sequence. The only shared obligations are the kernel invariants.
|
|
152
|
+
|
|
153
|
+
This separation solves the peer-skill routing problem: a strong host may correctly decide that a task primarily needs `tdd` or `security-review`; that choice is allowed. What it cannot silently erase is the routing/evidence/retry contract established by the runtime.
|
|
90
154
|
|
|
91
155
|
---
|
|
92
156
|
|
|
93
157
|
## Security Model & Data Locality
|
|
94
158
|
|
|
95
|
-
Harness runs with a
|
|
159
|
+
Harness runs with a local-first, zero-trust model:
|
|
160
|
+
|
|
161
|
+
1. **No Harness telemetry:** runtime scripts do not upload project code or state as a Harness service.
|
|
162
|
+
2. **Credential protection:** scripts do not require storing user secrets; interactive credentials remain human-controlled.
|
|
163
|
+
3. **Small hook surface:** hook scripts are local Node.js programs and should stay fast and auditable.
|
|
164
|
+
4. **Explicit enforcement labels:** unsupported/advisory behavior must not be documented as hard enforcement.
|
|
165
|
+
5. **Deterministic regression:** mechanism and routing behavior is tested with executable CI gates rather than inferred from documentation.
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## Validation Boundary
|
|
170
|
+
|
|
171
|
+
Static configuration is not enough to claim that a host behaves correctly. Validation is layered:
|
|
172
|
+
|
|
173
|
+
- syntax/reference/manifest checks prove package integrity,
|
|
174
|
+
- `ci/invariant-routing.test.js` proves the kernel contract and documentation invariants,
|
|
175
|
+
- `ci/doc-capability-consistency.test.js` prevents current platform capability claims from drifting apart,
|
|
176
|
+
- mechanism tests prove supported hook/plugin behavior,
|
|
177
|
+
- live host sessions are required before promoting a source/mechanism-tested adapter to live host enforcement.
|
|
96
178
|
|
|
97
|
-
|
|
98
|
-
2. **Credential Protection:** Harness never asks for or stores API keys or secrets. If any terminal command prompts for a password, Harness's rules immediately direct the model to halt and request manual entry by the human partner.
|
|
99
|
-
3. **Execution Gating:** The hook scripts are written in standard CommonJS, ensuring they compile and run fast (<200ms) to prevent blocking terminal operations.
|
|
179
|
+
The architecture and its Mermaid diagrams are part of that contract. Runtime changes that alter orchestration must update these documents in the same change.
|