harness-everything 0.3.6 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/plugins/marketplace.json +20 -0
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +8 -7
- package/.github/workflows/behavioral-evals.yml +3 -3
- package/.github/workflows/ci.yml +49 -28
- package/.github/workflows/release.yml +52 -27
- package/.nvmrc +1 -0
- package/.releaserc.json +113 -0
- package/AGENTS.md +44 -24
- package/BENCHMARK_SOP.md +38 -18
- package/CHANGELOG.md +196 -3
- package/CONTRIBUTING.md +37 -9
- package/Dockerfile +1 -1
- package/PRIVACY.md +25 -0
- package/README.md +126 -91
- package/RELEASING.md +93 -0
- package/SUPPORT.md +27 -0
- package/TERMS.md +21 -0
- package/VERIFICATION.md +192 -284
- package/behavioral-evals/README.md +21 -0
- package/behavioral-evals/case-validator.js +194 -0
- package/behavioral-evals/cases/baseline-environment-detection.yaml +21 -0
- package/behavioral-evals/cases/baseline-eval-harness.yaml +27 -0
- package/behavioral-evals/cases/baseline-fable-discipline.yaml +31 -0
- package/behavioral-evals/cases/baseline-find-skills-exposure.yaml +49 -0
- package/behavioral-evals/cases/baseline-find-skills.yaml +21 -0
- package/behavioral-evals/cases/baseline-git-commit.yaml +2 -2
- package/behavioral-evals/cases/baseline-install-cognitive-os.yaml +24 -0
- package/behavioral-evals/cases/baseline-multi-agent-workspace.yaml +28 -0
- package/behavioral-evals/cases/baseline-rewrite-commits.yaml +34 -0
- package/behavioral-evals/cases/baseline-self-evolve.yaml +31 -0
- package/behavioral-evals/cases/baseline-skill-creator.yaml +20 -0
- package/behavioral-evals/cases/baseline-skill-style.yaml +21 -0
- package/behavioral-evals/cases/baseline-to-spec.yaml +34 -0
- package/behavioral-evals/cases/baseline-to-tickets.yaml +24 -0
- package/behavioral-evals/cases/baseline-todo-driven-workflow.yaml +32 -0
- package/behavioral-evals/cases/baseline-using-git-worktrees.yaml +15 -0
- package/behavioral-evals/cases/breaker-zoom-out-after-3.yaml +2 -2
- package/behavioral-evals/cases/pressure-complexity-skip.yaml +2 -2
- package/behavioral-evals/cases/pressure-scope-bypass.yaml +7 -2
- package/behavioral-evals/cases/pressure-sunk-cost-retry.yaml +7 -4
- package/behavioral-evals/cases/scope-discipline.yaml +2 -1
- package/behavioral-evals/cases/verify-before-done.yaml +7 -5
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-debugging/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/historical-case.yaml +34 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-performance/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-security-review/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/baseline-simple-bugfix/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/grill-me-adversarial/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/live-run-blocker.md +16 -0
- package/behavioral-evals/evidence/2026-09-07/manifest.json +88 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/historical-case.yaml +28 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-scope-bypass/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/historical-case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-docs/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/historical-case.yaml +35 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-error-handling/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-23-pressure-skip-verification/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/historical-case.yaml +43 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-skip-verification--2026-08-27-pressure-skip-verification/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-23-pressure-sunk-cost-retry/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/case.yaml +50 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/historical-case.yaml +47 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/pressure-sunk-cost-retry--2026-08-27-pressure-sunk-cost-retry/result.sanitized.json +24 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/case.yaml +30 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/historical-case.yaml +23 -0
- package/behavioral-evals/evidence/2026-09-07/tdd-test-first/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.json +426 -0
- package/behavioral-evals/evidence/2026-09-07/triage-matrix.md +24 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/case.yaml +19 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/historical-case.yaml +18 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-claim-cites/provenance.json +30 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-23-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/historical-case.yaml +31 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-07/verify-before-done--2026-08-27-verify-before-done/result.sanitized.json +20 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/historical-case.yaml +33 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/provenance.json +42 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/result.sanitized.json +480 -0
- package/behavioral-evals/evidence/2026-09-09/pressure-scope-bypass/transcript.sanitized.jsonl +95 -0
- package/behavioral-evals/evidence-tool.js +443 -0
- package/behavioral-evals/fixtures/claude-denied.jsonl +3 -0
- package/behavioral-evals/fixtures/claude-legacy.json +1 -0
- package/behavioral-evals/fixtures/claude-oral-only.jsonl +2 -0
- package/behavioral-evals/fixtures/claude-success.jsonl +5 -0
- package/behavioral-evals/fixtures/claude-truncated.jsonl +3 -0
- package/behavioral-evals/fixtures/opencode-success.jsonl +6 -0
- package/behavioral-evals/results/2026-08-31-baseline-add-feature.json +20 -0
- package/behavioral-evals/results/2026-09-03-baseline-environment-detection-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-eval-harness-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-fable-discipline-pair.json +149 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-exposure-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-install-cognitive-os-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-multi-agent-workspace-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-rewrite-commits-pair.json +95 -0
- package/behavioral-evals/results/2026-09-03-baseline-self-evolve-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-creator-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-skill-style-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-spec-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-baseline-to-tickets-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-todo-driven-workflow-pair.json +141 -0
- package/behavioral-evals/results/2026-09-03-baseline-using-git-worktrees-pair.json +133 -0
- package/behavioral-evals/results/2026-09-03-summary-both.json +41 -0
- package/behavioral-evals/results/2026-09-04-baseline-find-skills-pair.json +141 -0
- package/behavioral-evals/results/2026-09-04-baseline-self-evolve-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-baseline-skill-creator-treatment.json +80 -0
- package/behavioral-evals/results/2026-09-04-summary-both.json +41 -0
- package/behavioral-evals/run-with-plugin.js +15 -31
- package/behavioral-evals/run.js +446 -105
- package/behavioral-evals/transcript-parser.js +591 -0
- package/bin/cli.js +24 -12
- package/ci/ab-test-cases/grill-me-adversarial.yaml +14 -4
- package/ci/ab-test-cases/tdd-test-first.yaml +13 -6
- package/ci/ab-test-cases/verification-loop-gates.yaml +20 -7
- package/ci/ab-test-cases/verify-before-claim-cites.yaml +7 -6
- package/ci/ab-test-harness.js +113 -38
- package/ci/action-gate-platform-doc.test.js +31 -0
- package/ci/action-gate.test.js +260 -0
- package/ci/cases/case2.json +2 -2
- package/ci/consistency-check.js +51 -5
- package/ci/description-collision.js +15 -6
- package/ci/disclosure-check.js +225 -0
- package/ci/doc-capability-consistency.test.js +180 -0
- package/ci/fixtures/phase4/parallel-disjoint.stages.json +32 -0
- package/ci/fixtures/phase4/parallel-read-only.stages.json +32 -0
- package/ci/installer-roundtrip.test.js +159 -0
- package/ci/invariant-routing.test.js +74 -0
- package/ci/issue99-runtime.test.js +218 -0
- package/ci/mechanism-2aa-codex-action-gate-policy.test.js +117 -0
- package/ci/mechanism-2ab-openai-plugin-sync-integrity.test.js +80 -0
- package/ci/mechanism-2j-behavioral-runner.test.js +43 -1
- package/ci/mechanism-2k-reference-check.test.js +57 -2
- package/ci/mechanism-2l-multi-agent-workspace.test.js +202 -6
- package/ci/mechanism-2m-self-repo-guard.test.js +65 -0
- package/ci/mechanism-2n-opencode-plugin.test.js +461 -0
- package/ci/mechanism-2o-state-home-isolation.test.js +183 -0
- package/ci/mechanism-2p-installer-links.test.js +183 -0
- package/ci/mechanism-2q-installer-global-shared-store.test.js +132 -0
- package/ci/mechanism-2r-uninstall-flags.test.js +148 -0
- package/ci/mechanism-2s-tdd-quality.test.js +203 -0
- package/ci/mechanism-2t-disclosure.test.js +90 -0
- package/ci/mechanism-2t-evidence-triage.test.js +200 -0
- package/ci/mechanism-2t-issue-20.test.js +75 -0
- package/ci/mechanism-2t-platform-skill-targets.test.js +164 -0
- package/ci/mechanism-2t-transcript-parser.test.js +179 -0
- package/ci/mechanism-2t-workspace-identity.test.js +200 -0
- package/ci/mechanism-2u-issue-42-regressions.test.js +239 -0
- package/ci/mechanism-2u-issue-44.test.js +35 -0
- package/ci/mechanism-2v-verifier-boundaries.test.js +138 -0
- package/ci/mechanism-2w-platform-compatibility.test.js +148 -0
- package/ci/mechanism-2x-release-automation.test.js +157 -0
- package/ci/mechanism-2x-release-commit-message.test.js +66 -0
- package/ci/mechanism-2y-release-tooling-compat.test.js +47 -0
- package/ci/mechanism-2z-claude-plugin-manifest.test.js +29 -0
- package/ci/mutation-check.js +53 -0
- package/ci/mutation-loader.js +57 -0
- package/ci/negative-controls.js +76 -46
- package/ci/openai-plugin-package.test.js +182 -0
- package/ci/openai-plugin-runtime.test.js +175 -0
- package/ci/openai-plugin-submission.test.js +101 -0
- package/ci/phase4-ensemble-memory.test.js +307 -0
- package/ci/reference-check.js +73 -25
- package/ci/release-consistency-check.js +24 -1
- package/ci/repository-contract.test.js +17 -0
- package/ci/router-phase2-safety.test.js +81 -0
- package/ci/router-workflow-plan.test.js +264 -0
- package/ci/routing-check.js +6 -5
- package/ci/runner.js +6 -5
- package/ci/skill-routing-check.js +48 -11
- package/ci/test-behavioral-case-validation.js +62 -0
- package/ci/test-helper.js +10 -0
- package/ci/workflow-plan-consumers-edge.test.js +72 -0
- package/ci/workflow-plan-consumers.test.js +276 -0
- package/docs/architecture.md +137 -57
- package/docs/audit.md +22 -20
- package/docs/codex-action-gate.md +90 -0
- package/docs/issue-20-rollup.md +47 -0
- package/docs/mechanism-first-skill-mesh.md +22 -20
- package/docs/openai-plugin.md +118 -0
- package/docs/philosophy.md +28 -17
- package/docs/platform-capabilities.md +73 -0
- package/docs/platform-compatibility.json +270 -0
- package/docs/reflection.md +37 -22
- package/docs/release-evidence.md +27 -0
- package/docs/repository-contract.md +27 -0
- package/docs/routing.md +157 -44
- package/docs/security-audit.md +38 -0
- package/docs/troubleshooting.md +94 -51
- package/docs/verifier-boundaries.md +45 -0
- package/docs/workflows/fable-mode.md +77 -21
- package/docs/workflows/find-skills.md +40 -0
- package/docs/workflows/grill-with-docs.md +2 -2
- package/docs/workflows/multi-agent-workspace.md +22 -10
- package/docs/workflows/self-evolve.md +33 -0
- package/docs/workflows/tdd.md +36 -0
- package/docs/workflows/todo-driven-workflow.md +33 -0
- package/environment-detection/SKILL.md +3 -3
- package/eval-harness/SKILL.md +7 -7
- package/eval-harness/scripts/evaluate.js +12 -10
- package/evals/evaluation-reports.md +14 -0
- package/evals/harness-everything/tasks/positive-tier1.yaml +1 -1
- package/evals/harness-everything/tasks/positive-tier2.yaml +1 -1
- package/evals/harness-everything/tasks/positive-tier3.yaml +1 -1
- package/evals/install-cognitive-os/tasks/positive-trigger.yaml +1 -1
- package/fable-discipline/SKILL.md +2 -2
- package/fable-mode/CONTRACT-FORMAT.md +147 -33
- package/fable-mode/SKILL.md +3 -3
- package/fable-mode/agents/fable-orchestrator.md +95 -35
- package/fable-mode/execution-guardrails/SKILL.md +2 -2
- package/fable-mode/fable-haiku/SKILL.md +3 -3
- package/fable-mode/fable-opus/SKILL.md +3 -3
- package/fable-mode/fable-sonnet/SKILL.md +3 -3
- package/fable-mode/scripts/ensemble-review.js +371 -0
- package/fable-mode/scripts/model-selector.js +134 -3
- package/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/find-skills/SKILL.md +5 -5
- package/find-skills/references/discovery-flow.md +1 -1
- package/find-skills/scripts/use-skill.js +7 -1
- package/git-commit/SKILL.md +4 -11
- package/grill-me/SKILL.md +5 -5
- package/grill-with-docs/SKILL.md +7 -7
- package/grill-with-docs/references/session-playbook.md +6 -4
- package/harness-everything/SKILL.md +23 -22
- package/harness-everything/references/router-workflow-plan.md +128 -0
- package/harness-everything/references/triage-and-tiers.md +173 -96
- package/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/harness-everything/scripts/bootstrap.js +13 -4
- package/harness-everything/scripts/ensemble-policy.js +102 -0
- package/harness-everything/scripts/kernel-router.js +212 -0
- package/harness-everything/scripts/router-contract.js +612 -0
- package/harness-everything/scripts/self-heal.js +29 -20
- package/harness-everything/scripts/tier-router.js +346 -221
- package/hooks/hooks.json +60 -15
- package/hooks/scripts/action-gate-rules.json +77 -0
- package/hooks/scripts/action-gate.js +528 -0
- package/hooks/scripts/atomic-commit-check.js +5 -5
- package/hooks/scripts/codex-action-gate-post.js +46 -0
- package/hooks/scripts/codex-permission-request.js +140 -0
- package/hooks/scripts/context-compact.js +4 -3
- package/hooks/scripts/contract-test.js +118 -51
- package/hooks/scripts/lib/fable-contracts.js +121 -0
- package/hooks/scripts/lib/harness-state.js +164 -28
- package/hooks/scripts/lib/platforms/claude.js +45 -62
- package/hooks/scripts/lib/platforms/codex.js +35 -61
- package/hooks/scripts/lib/platforms/continue.js +7 -7
- package/hooks/scripts/lib/platforms/copilot.js +29 -53
- package/hooks/scripts/lib/platforms/hermes.js +71 -20
- package/hooks/scripts/openai-session-start.js +81 -0
- package/hooks/scripts/rule-of-3-tracker.js +9 -8
- package/hooks/scripts/rule-of-3.js +29 -21
- package/hooks/scripts/state-persist.js +5 -4
- package/hooks/scripts/stop-gate.js +2 -2
- package/hooks/scripts/subagent-scope-guard.js +144 -38
- package/hooks/scripts/subagent-start.js +19 -0
- package/hooks/scripts/subagent-stop.js +6 -0
- package/improve-codebase-architecture/SKILL.md +2 -4
- package/improve-codebase-architecture/references/README.md +16 -0
- package/install-cognitive-os/SKILL.md +28 -22
- package/multi-agent-workspace/SKILL.md +16 -15
- package/multi-agent-workspace/references/architecture-guide.md +13 -9
- package/multi-agent-workspace/references/orchestration.md +39 -22
- package/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/multi-agent-workspace/scripts/index_memory.js +79 -26
- package/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/multi-agent-workspace/scripts/scaffold.js +258 -103
- package/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/multi-agent-workspace/workflows/03-generation.md +8 -10
- package/multi-agent-workspace/workflows/04-launcher.md +5 -5
- package/opencode-plugin/README.md +114 -71
- package/opencode-plugin/hooks/verify.js +62 -12
- package/opencode-plugin/index.mjs +547 -0
- package/opencode-plugin/plugin.json +4 -12
- package/package.json +20 -4
- package/plugins/harness-everything/.codex-plugin/plugin.json +39 -0
- package/plugins/harness-everything/hooks/hooks.json +197 -0
- package/plugins/harness-everything/hooks/scripts/action-gate-rules.json +77 -0
- package/plugins/harness-everything/hooks/scripts/action-gate.js +528 -0
- package/plugins/harness-everything/hooks/scripts/atomic-commit-check.js +68 -0
- package/plugins/harness-everything/hooks/scripts/boundary-guard.js +55 -0
- package/plugins/harness-everything/hooks/scripts/codex-action-gate-post.js +46 -0
- package/plugins/harness-everything/hooks/scripts/codex-permission-request.js +140 -0
- package/plugins/harness-everything/hooks/scripts/context-compact.js +115 -0
- package/plugins/harness-everything/hooks/scripts/contract-test.js +146 -0
- package/plugins/harness-everything/hooks/scripts/depth-guard.js +65 -0
- package/plugins/harness-everything/hooks/scripts/lib/fable-contracts.js +121 -0
- package/plugins/harness-everything/hooks/scripts/lib/harness-state.js +275 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/claude.js +168 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/codex.js +88 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/continue.js +91 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/copilot.js +85 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/cursor.js +89 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/hermes.js +108 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/index.js +17 -0
- package/plugins/harness-everything/hooks/scripts/lib/platforms/worktrees.js +20 -0
- package/plugins/harness-everything/hooks/scripts/lib/workspace.js +290 -0
- package/plugins/harness-everything/hooks/scripts/openai-session-start.js +81 -0
- package/plugins/harness-everything/hooks/scripts/reset-circuit-breaker.js +30 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3-tracker.js +125 -0
- package/plugins/harness-everything/hooks/scripts/rule-of-3.js +195 -0
- package/plugins/harness-everything/hooks/scripts/state-persist.js +100 -0
- package/plugins/harness-everything/hooks/scripts/stop-gate.js +80 -0
- package/plugins/harness-everything/hooks/scripts/subagent-scope-guard.js +194 -0
- package/plugins/harness-everything/hooks/scripts/subagent-start.js +19 -0
- package/plugins/harness-everything/hooks/scripts/subagent-stop.js +6 -0
- package/plugins/harness-everything/hooks/session-start.js +81 -0
- package/plugins/harness-everything/plugin.json +43 -0
- package/plugins/harness-everything/scripts/lib/advisory-text.js +174 -0
- package/plugins/harness-everything/skills/environment-detection/SKILL.md +41 -0
- package/plugins/harness-everything/skills/environment-detection/references/shell-syntax-rules.md +76 -0
- package/plugins/harness-everything/skills/environment-detection/scripts/preflight.js +103 -0
- package/plugins/harness-everything/skills/eval-harness/SKILL.md +42 -0
- package/plugins/harness-everything/skills/eval-harness/references/scoring-rubric.md +68 -0
- package/plugins/harness-everything/skills/eval-harness/scripts/evaluate.js +79 -0
- package/plugins/harness-everything/skills/fable-discipline/SKILL.md +41 -0
- package/plugins/harness-everything/skills/fable-discipline/references/discipline-rules.md +22 -0
- package/plugins/harness-everything/skills/fable-mode/CONTRACT-FORMAT.md +165 -0
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +159 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-verifier.md +32 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-haiku.md +52 -0
- package/plugins/harness-everything/skills/fable-mode/agents/fable-worker-sonnet.md +53 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +43 -0
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/references/guardrail-rules.md +56 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/references/routing-notes.md +26 -0
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +36 -0
- package/plugins/harness-everything/skills/fable-mode/model-matrix.json +32 -0
- package/plugins/harness-everything/skills/fable-mode/references/execution-phases.md +37 -0
- package/plugins/harness-everything/skills/fable-mode/references/model-matrix.md +39 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/ensemble-review.js +371 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/model-selector.js +260 -0
- package/plugins/harness-everything/skills/fable-mode/scripts/workflow-plan-consumer.js +337 -0
- package/plugins/harness-everything/skills/find-skills/SKILL.md +48 -0
- package/plugins/harness-everything/skills/find-skills/references/discovery-flow.md +102 -0
- package/plugins/harness-everything/skills/find-skills/scripts/use-skill.js +112 -0
- package/plugins/harness-everything/skills/git-commit/SKILL.md +38 -0
- package/plugins/harness-everything/skills/git-commit/guides/ANGULAR_STYLE.md +61 -0
- package/plugins/harness-everything/skills/git-commit/guides/COMMIT_GENERATION.md +24 -0
- package/plugins/harness-everything/skills/git-commit/guides/LANGUAGE_DETECTION.md +16 -0
- package/plugins/harness-everything/skills/git-commit/guides/MAIN_REPO.md +15 -0
- package/plugins/harness-everything/skills/git-commit/guides/SUBMODULES.md +27 -0
- package/plugins/harness-everything/skills/git-commit/references/commit-flow.md +30 -0
- package/plugins/harness-everything/skills/grill-me/SKILL.md +43 -0
- package/plugins/harness-everything/skills/grill-me/references/grilling-playbook.md +38 -0
- package/plugins/harness-everything/skills/grill-with-docs/ADR-FORMAT.md +47 -0
- package/plugins/harness-everything/skills/grill-with-docs/CONTEXT-FORMAT.md +63 -0
- package/plugins/harness-everything/skills/grill-with-docs/SKILL.md +41 -0
- package/plugins/harness-everything/skills/grill-with-docs/references/session-playbook.md +99 -0
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +43 -0
- package/plugins/harness-everything/skills/harness-everything/examples/README.md +9 -0
- package/plugins/harness-everything/skills/harness-everything/examples/todo-workflow.md +22 -0
- package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md +128 -0
- package/plugins/harness-everything/skills/harness-everything/references/skill-registry.md +39 -0
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +193 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-task-shape.schema.json +126 -0
- package/plugins/harness-everything/skills/harness-everything/schemas/router-workflow-plan.schema.json +149 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/bootstrap.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/ensemble-policy.js +102 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +212 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/router-contract.js +612 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/routing-keywords.json +320 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/self-heal.js +250 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +459 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/verify-gate.js +100 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/SKILL.md +38 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/DEEPENING.md +37 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/HTML-REPORT.md +123 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/INTERFACE-DESIGN.md +44 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/LANGUAGE.md +53 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/guides/refactoring-workflow.md +59 -0
- package/plugins/harness-everything/skills/improve-codebase-architecture/references/README.md +16 -0
- package/plugins/harness-everything/skills/install-cognitive-os/SKILL.md +44 -0
- package/plugins/harness-everything/skills/install-cognitive-os/references/cognitive-loop.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/NOTICE-agency-agents.md +27 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/csharp-monorepo.md +123 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/examples/typescript-cli.md +126 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-claude.md +77 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-codex.md +146 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-continue.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-copilot.md +74 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-cursor.md +40 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/platform-hermes.md +38 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/guidelines/role-capabilities.md +108 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/agency-agents.md +31 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/architecture-guide.md +36 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/references/orchestration.md +65 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/agency-catalog.js +190 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/consume-workflow-plan.js +129 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/index_memory.js +104 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/path-boundary.js +59 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/project-docs-resolver.js +357 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/scripts/scaffold.js +408 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/AGENTS.md +20 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/README.md +17 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/agent-launcher.md +68 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-csharp.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-polyglot.md +62 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/backend-developer-ts.md +57 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/memory-keeper.md +37 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/requirement-analyzer.md +66 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/specialist-agent.md +34 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/task-verifier.md +56 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/templates/tool-maker.md +33 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/01-init.md +22 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/02-analysis.md +23 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/03-generation.md +25 -0
- package/plugins/harness-everything/skills/multi-agent-workspace/workflows/04-launcher.md +16 -0
- package/plugins/harness-everything/skills/repo-docs/SKILL.md +42 -0
- package/plugins/harness-everything/skills/repo-docs/references/process-guide.md +60 -0
- package/plugins/harness-everything/skills/repo-docs/templates/agents-template.md +264 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-agents-template.md +50 -0
- package/plugins/harness-everything/skills/repo-docs/templates/knowledge-readme-template.md +83 -0
- package/plugins/harness-everything/skills/repo-docs/templates/manual-recon.md +117 -0
- package/plugins/harness-everything/skills/repo-docs/templates/multi-skills-readme-template.md +72 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-agents-template.md +78 -0
- package/plugins/harness-everything/skills/repo-docs/templates/product-readme-template.md +95 -0
- package/plugins/harness-everything/skills/repo-docs/templates/readme-template.md +136 -0
- package/plugins/harness-everything/skills/repo-docs/templates/single-skill-readme-template.md +70 -0
- package/plugins/harness-everything/skills/repo-docs/templates/skills-agents-template.md +60 -0
- package/plugins/harness-everything/skills/rewrite-commits/SKILL.md +37 -0
- package/plugins/harness-everything/skills/rewrite-commits/references/workflow.md +18 -0
- package/plugins/harness-everything/skills/security-review/SKILL.md +47 -0
- package/plugins/harness-everything/skills/security-review/guides/OWASP-PATTERNS.md +31 -0
- package/plugins/harness-everything/skills/security-review/guides/STRIDE-THREAT-MODEL.md +21 -0
- package/plugins/harness-everything/skills/security-review/references/security-checklist.md +478 -0
- package/plugins/harness-everything/skills/security-review/scripts/audit-secrets.js +85 -0
- package/plugins/harness-everything/skills/self-evolve/SKILL.md +45 -0
- package/plugins/harness-everything/skills/self-evolve/references/memory-resolution.md +61 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/persist-memory.js +272 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/register-dynamic-skill.js +253 -0
- package/plugins/harness-everything/skills/self-evolve/scripts/self-regression.js +125 -0
- package/plugins/harness-everything/skills/skill-creator/SKILL.md +47 -0
- package/plugins/harness-everything/skills/skill-creator/references/authoring-workflow.md +14 -0
- package/plugins/harness-everything/skills/skill-creator/references/dynamic-generation-contract.md +21 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-checklist.md +16 -0
- package/plugins/harness-everything/skills/skill-creator/references/quality-principles.md +81 -0
- package/plugins/harness-everything/skills/skill-creator/references/testing-workflow.md +27 -0
- package/plugins/harness-everything/skills/skill-style/SKILL.md +42 -0
- package/plugins/harness-everything/skills/skill-style/references/style-guide.md +62 -0
- package/plugins/harness-everything/skills/tdd/SKILL.md +38 -0
- package/plugins/harness-everything/skills/tdd/fixtures/pass-evidence.json +100 -0
- package/plugins/harness-everything/skills/tdd/fixtures/quality-scenarios.json +20 -0
- package/plugins/harness-everything/skills/tdd/guides/deep-modules.md +33 -0
- package/plugins/harness-everything/skills/tdd/guides/interface-design.md +31 -0
- package/plugins/harness-everything/skills/tdd/guides/mocking.md +59 -0
- package/plugins/harness-everything/skills/tdd/guides/refactoring.md +10 -0
- package/plugins/harness-everything/skills/tdd/guides/tests.md +61 -0
- package/plugins/harness-everything/skills/tdd/references/core-discipline.md +51 -0
- package/plugins/harness-everything/skills/tdd/references/integration-testing.md +13 -0
- package/plugins/harness-everything/skills/tdd/references/quality-model.md +68 -0
- package/plugins/harness-everything/skills/tdd/references/unit-testing.md +15 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-evidence.schema.json +211 -0
- package/plugins/harness-everything/skills/tdd/schemas/quality-report.schema.json +110 -0
- package/plugins/harness-everything/skills/tdd/scripts/quality-gate.js +475 -0
- package/plugins/harness-everything/skills/to-spec/SKILL.md +40 -0
- package/plugins/harness-everything/skills/to-spec/references/process.md +77 -0
- package/plugins/harness-everything/skills/to-spec/scripts/check-project-docs.js +239 -0
- package/plugins/harness-everything/skills/to-spec/scripts/project-docs-resolver.js +111 -0
- package/plugins/harness-everything/skills/to-spec/templates/cli-reference.md +31 -0
- package/plugins/harness-everything/skills/to-spec/templates/dev-doc.md +27 -0
- package/plugins/harness-everything/skills/to-spec/templates/feature-spec.md +59 -0
- package/plugins/harness-everything/skills/to-spec/templates/schema-doc.md +31 -0
- package/plugins/harness-everything/skills/to-tickets/SKILL.md +42 -0
- package/plugins/harness-everything/skills/to-tickets/references/publishing-and-templates.md +97 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/SKILL.md +42 -0
- package/plugins/harness-everything/skills/todo-driven-workflow/references/execution-guide.md +27 -0
- package/plugins/harness-everything/skills/using-git-worktrees/SKILL.md +49 -0
- package/plugins/harness-everything/skills/using-git-worktrees/references/workflow-details.md +108 -0
- package/plugins/harness-everything/skills/verification-loop/SKILL.md +46 -0
- package/plugins/harness-everything/skills/verification-loop/references/verification-phases.md +116 -0
- package/plugins/harness-everything/skills/verification-loop/templates/verification-report.template.md +15 -0
- package/plugins/harness-everything/skills/verify-before-claim/SKILL.md +43 -0
- package/plugins/harness-everything/skills/verify-before-claim/references/verification-guide.md +73 -0
- package/plugins/harness-everything/skills/zoom-out/SKILL.md +42 -0
- package/plugins/harness-everything/skills/zoom-out/references/circuit-breaker.md +66 -0
- package/plugins/harness-everything/skills/zoom-out/templates/zoom-out-report.template.md +20 -0
- package/repo-docs/SKILL.md +3 -7
- package/rewrite-commits/SKILL.md +2 -2
- package/scripts/build-openai-submission.js +158 -0
- package/scripts/installer.js +86 -11
- package/scripts/lib/advisory-text.js +25 -18
- package/scripts/lib/execution-contract.js +70 -0
- package/scripts/lib/manifest.js +34 -38
- package/scripts/lib/path-boundary.js +90 -0
- package/scripts/lib/skills.js +124 -70
- package/scripts/lib/workspace.js +269 -7
- package/scripts/repository-contract.js +213 -0
- package/scripts/sync-current-docs.js +12 -0
- package/scripts/sync-openai-plugin.js +91 -0
- package/scripts/sync-release-version.js +212 -0
- package/security-review/SKILL.md +4 -7
- package/security-review/scripts/audit-secrets.js +9 -8
- package/self-evolve/SKILL.md +9 -11
- package/self-evolve/references/memory-resolution.md +4 -1
- package/self-evolve/scripts/persist-memory.js +221 -88
- package/self-evolve/scripts/register-dynamic-skill.js +12 -10
- package/self-evolve/scripts/self-regression.js +23 -8
- package/skill-creator/SKILL.md +12 -14
- package/skill-creator/references/quality-checklist.md +1 -0
- package/skill-style/SKILL.md +4 -4
- package/skill-style/references/style-guide.md +29 -3
- package/submission/openai/README.md +67 -0
- package/submission/openai/listing.json +31 -0
- package/submission/openai/test-cases.json +61 -0
- package/tdd/SKILL.md +20 -26
- package/tdd/fixtures/pass-evidence.json +100 -0
- package/tdd/fixtures/quality-scenarios.json +20 -0
- package/tdd/references/core-discipline.md +2 -0
- package/tdd/references/integration-testing.md +13 -0
- package/tdd/references/quality-model.md +68 -0
- package/tdd/references/unit-testing.md +15 -0
- package/tdd/schemas/quality-evidence.schema.json +211 -0
- package/tdd/schemas/quality-report.schema.json +110 -0
- package/tdd/scripts/quality-gate.js +475 -0
- package/to-spec/SKILL.md +3 -5
- package/to-spec/scripts/check-project-docs.js +42 -62
- package/to-spec/scripts/project-docs-resolver.js +111 -0
- package/to-tickets/SKILL.md +4 -4
- package/todo-driven-workflow/SKILL.md +6 -4
- package/using-git-worktrees/SKILL.md +3 -3
- package/verification-loop/SKILL.md +4 -4
- package/verify-before-claim/SKILL.md +2 -2
- package/zoom-out/SKILL.md +4 -4
- package/ab-test-deep-analysis-2026-08-27.md +0 -383
- package/ab-test-results-2026-08-27.md +0 -152
- package/ab-test-statistical-analysis-31cases.md +0 -193
- package/opencode-plugin/hooks/circuit-breaker.js +0 -121
- package/opencode-plugin/hooks/compliance.js +0 -128
- package/opencode-plugin/hooks/post-edit.js +0 -62
- package/opencode-plugin/hooks/pre-complete.js +0 -65
- package/opencode-plugin/test-hook-firing.js +0 -198
|
@@ -1,29 +1,85 @@
|
|
|
1
|
-
# Workflow: Fable Mode
|
|
1
|
+
# Workflow: Fable Mode (v3)
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> Stage large, multi-source or multi-session work through an explicit stage map, named agents, failable per-stage checks, and skeptical delivery review. Fable is not the default path for ordinary Tier 2 work.
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
-
## 1.
|
|
7
|
+
## 1. When Fable Mode Applies
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
Use `fable-mode` when the work is large enough that one undifferentiated execution loop would lose scope or evidence: multi-file architectural work, multi-source investigation, multi-session execution, or an explicit `fable on haiku|sonnet|opus` request.
|
|
10
10
|
|
|
11
11
|
```mermaid
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
Delegate -->|No Sub-agent Tool| InlinePersona["Inline Persona Role-Switch Fallback"]
|
|
19
|
-
|
|
20
|
-
SpawnSub --> ScopeCheck["Monitor via hooks/scripts/subagent-scope-guard.js"]
|
|
21
|
-
InlinePersona --> ScopeCheck
|
|
22
|
-
|
|
23
|
-
ScopeCheck --> VerifyStep["4. Integration Verification via harness-everything/scripts/verify-gate.js"]
|
|
24
|
-
VerifyStep --> CheckGate{Integration Verification Passed?}
|
|
25
|
-
|
|
26
|
-
CheckGate -->|No: Integration Error| FixBlocker["Add Blocker Item to todo-driven-workflow & Fix"] --> VerifyStep
|
|
27
|
-
CheckGate -->|Yes: Gate Passes| Transition["5. Transition Scaffolding to TDD Iteration"]
|
|
28
|
-
Transition --> End([Macro Scaffolding Successfully Deployed])
|
|
12
|
+
flowchart TD
|
|
13
|
+
Request[User request] --> Scope{Large / multi-source / multi-session?}
|
|
14
|
+
Scope -->|No| Normal[Use the normal Harness execution path]
|
|
15
|
+
Scope -->|Yes| Discover[Discover runtime + authorized file scope]
|
|
16
|
+
Discover --> Map[Write numbered stage map]
|
|
17
|
+
Map --> Run[Execute one stage at a time]
|
|
29
18
|
```
|
|
19
|
+
|
|
20
|
+
`fable-mode` does **not** imply a mandatory `todo-driven-workflow → TDD` pipeline. Each stage chooses only the tactics and domain skills needed for its own artifact and pass condition.
|
|
21
|
+
|
|
22
|
+
## 2. Stage Contract
|
|
23
|
+
|
|
24
|
+
Every stage owns one concrete artifact and one explicit pass condition. A stage is not complete because an agent says it is complete; its named check must pass.
|
|
25
|
+
|
|
26
|
+
```mermaid
|
|
27
|
+
stateDiagram-v2
|
|
28
|
+
[*] --> Planned
|
|
29
|
+
Planned --> Running: artifact + pass condition defined
|
|
30
|
+
Running --> Verifying: stage work produced
|
|
31
|
+
Verifying --> Passed: named check passes
|
|
32
|
+
Verifying --> Blocked: check fails / unresolved dependency
|
|
33
|
+
Blocked --> Running: blocker resolved
|
|
34
|
+
Passed --> Reviewed: cold review when risk warrants it
|
|
35
|
+
Reviewed --> Recorded: audit record written
|
|
36
|
+
Recorded --> [*]
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
The stage record preserves the requested/effective model, fallback reason, stage brief, artifact, pass condition, verification command, and verifier result.
|
|
40
|
+
|
|
41
|
+
## 3. Model Selection and Delegation
|
|
42
|
+
|
|
43
|
+
Explicit model requests are resolved through `model-selector.js`. Fable never silently downgrades a requested model.
|
|
44
|
+
|
|
45
|
+
```mermaid
|
|
46
|
+
flowchart LR
|
|
47
|
+
Requested[Requested model] --> Selector[model-selector.js]
|
|
48
|
+
Selector -->|Available| Named[Named fable agent]
|
|
49
|
+
Selector -->|Fallback allowed| Fallback[Recorded fallback]
|
|
50
|
+
Selector -->|No valid fallback| Blocked[Escalate blocked stage]
|
|
51
|
+
Named --> Stage[Bounded stage brief]
|
|
52
|
+
Fallback --> Stage
|
|
53
|
+
Stage --> Verify[Named stage check]
|
|
54
|
+
Verify -->|High-stakes artifact| Cold[fable-verifier cold review]
|
|
55
|
+
Verify -->|Ordinary artifact| Record[Record evidence]
|
|
56
|
+
Cold --> Record
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
The intended role split is explicit rather than magical: Opus is suitable for orchestration, Sonnet for reasoning-heavy stage work, and Haiku for bounded mechanical work. Workers do not spawn workers.
|
|
60
|
+
|
|
61
|
+
## 4. Scope and Context Discipline
|
|
62
|
+
|
|
63
|
+
`fable-discipline` acts as the shadow guard while Fable is active. A worker receives the current stage contract and authorized scope, not the entire unresolved history.
|
|
64
|
+
|
|
65
|
+
```mermaid
|
|
66
|
+
flowchart TD
|
|
67
|
+
Stage[Current stage contract] --> Worker[Named worker]
|
|
68
|
+
Worker --> Scope{Stayed inside authorized scope?}
|
|
69
|
+
Scope -->|No| Block[Stop + report scope violation]
|
|
70
|
+
Scope -->|Yes| Check[Run stage verification]
|
|
71
|
+
Check -->|Fail| Diagnose[Record blocker / diagnose]
|
|
72
|
+
Diagnose --> Replan{Need full replan?}
|
|
73
|
+
Replan -->|No| Worker
|
|
74
|
+
Replan -->|Yes, max two| Map[Update stage map]
|
|
75
|
+
Map --> Worker
|
|
76
|
+
Check -->|Pass| Handoff[Compact evidence + handoff]
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
At most two full replans are allowed before unresolved blockers are escalated rather than hidden behind endless restructuring.
|
|
80
|
+
|
|
81
|
+
## 5. Completion Boundary
|
|
82
|
+
|
|
83
|
+
Fable completion means the planned stages have produced their artifacts, their checks have passed, and high-risk outputs have received skeptical review where required. It does not mean every possible companion skill was invoked.
|
|
84
|
+
|
|
85
|
+
The canonical executable and policy details live in `fable-mode/SKILL.md`, `fable-mode/model-matrix.json`, `fable-mode/scripts/model-selector.js`, and `fable-mode/references/model-matrix.md`.
|
|
@@ -34,3 +34,43 @@ flowchart TD
|
|
|
34
34
|
If search, network, or application fails, report the failure and continue with
|
|
35
35
|
the direct built-in solution. Never treat an unaudited third-party skill as a
|
|
36
36
|
first-party Harness skill or add it to the static manifest.
|
|
37
|
+
|
|
38
|
+
## Lazy-loading and trust boundary
|
|
39
|
+
|
|
40
|
+
The local catalog is the first source of truth. A candidate is inspected
|
|
41
|
+
before approval, and the detailed body is loaded only after the user agrees
|
|
42
|
+
to apply it.
|
|
43
|
+
|
|
44
|
+
```mermaid
|
|
45
|
+
sequenceDiagram
|
|
46
|
+
participant Agent
|
|
47
|
+
participant Catalog as Local catalog
|
|
48
|
+
participant Search as skills.sh / npx
|
|
49
|
+
participant User
|
|
50
|
+
Agent->>Catalog: Check existing coverage
|
|
51
|
+
alt Covered locally
|
|
52
|
+
Catalog-->>Agent: Return the local skill
|
|
53
|
+
else Missing locally
|
|
54
|
+
Agent->>Search: Find candidates
|
|
55
|
+
Search-->>Agent: Return source and metadata
|
|
56
|
+
Agent->>User: Present candidate for approval
|
|
57
|
+
User-->>Agent: Approve or decline
|
|
58
|
+
Agent->>Agent: Load body only after approval
|
|
59
|
+
end
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Failure handling
|
|
63
|
+
|
|
64
|
+
Search and application are optional edges. A failed edge returns control to a
|
|
65
|
+
direct solution without changing the static catalog.
|
|
66
|
+
|
|
67
|
+
```mermaid
|
|
68
|
+
flowchart TD
|
|
69
|
+
Search[Search candidate] --> Inspect[Inspect source and trust signal]
|
|
70
|
+
Inspect -->|Unsafe or unclear| Decline[Decline and use built-in path]
|
|
71
|
+
Inspect -->|Acceptable| Approval{Human approval?}
|
|
72
|
+
Approval -->|No| Decline
|
|
73
|
+
Approval -->|Yes| Apply[Apply for the session]
|
|
74
|
+
Apply -->|Failure| Decline
|
|
75
|
+
Apply -->|Success| Verify[Verify the applied skill]
|
|
76
|
+
```
|
|
@@ -19,10 +19,10 @@ graph TD
|
|
|
19
19
|
UpdateGlossary --> ResolveADRPath{Resolve ADR Storage Path}
|
|
20
20
|
|
|
21
21
|
ResolveADRPath -->|docs/adr/ or CONTEXT-MAP.md Exists| WriteDocs["Write ADR to docs/adr/ or CONTEXT-MAP.md Location"]
|
|
22
|
-
ResolveADRPath -->|No Folder Found|
|
|
22
|
+
ResolveADRPath -->|No Folder Found| WriteFallback["Write to committable docs/adr"]
|
|
23
23
|
|
|
24
24
|
WriteDocs --> ToSpec["Hand off to to-spec for Outline Preview & Spec/ADR Publishing"]
|
|
25
|
-
|
|
25
|
+
WriteFallback --> ToSpec
|
|
26
26
|
ToSpec --> End([Route to to-tickets / fable-mode / tdd])
|
|
27
27
|
```
|
|
28
28
|
|
|
@@ -9,17 +9,28 @@ Discover target + source
|
|
|
9
9
|
|
|
|
10
10
|
Validate divisions, frontmatter, names, slugs, revision, platform
|
|
11
11
|
|
|
|
12
|
-
|
|
13
|
-
state / logs / decisions / domain / architecture / roles
|
|
12
|
+
Resolve global runtime + repository document paths
|
|
14
13
|
|
|
|
15
|
-
Write
|
|
14
|
+
Write global manifest/handoff + selected roles
|
|
16
15
|
|
|
|
17
|
-
Run
|
|
16
|
+
Run installed immutable index_memory.js -> global memory-index.md
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
```mermaid
|
|
20
|
+
flowchart TD
|
|
21
|
+
Source[Optional agency source] --> Validate[Validate metadata and selection]
|
|
22
|
+
Validate --> Resolve[Shared project-docs resolver]
|
|
23
|
+
Resolve --> Runtime[Global workspace-keyed runtime]
|
|
24
|
+
Resolve --> Docs[Committable decision/domain/architecture paths]
|
|
25
|
+
Runtime --> Router[Installed immutable skill]
|
|
26
|
+
Runtime --> Indexer[Installed immutable indexer]
|
|
27
|
+
Router --> Handoff[Manifest and handoff]
|
|
28
|
+
Indexer --> Memory[Global memory index]
|
|
18
29
|
```
|
|
19
30
|
|
|
20
31
|
## Selection contract
|
|
21
32
|
|
|
22
|
-
Use `--agency-source <path>` or `AGENCY_AGENTS_SOURCE`. Select by repeated
|
|
33
|
+
Use `--workspace <path>` plus `--agency-source <path>` or `AGENCY_AGENTS_SOURCE`. Select by repeated
|
|
23
34
|
`--division` or `--agent`; `--all-agents` is explicit and metadata-only. The
|
|
24
35
|
generated launcher lists each selected role's division, source-relative file,
|
|
25
36
|
description, and boundary. Specialist bodies are loaded only on demand after a
|
|
@@ -31,12 +42,13 @@ No source creates a visible `agency.status: unavailable` fallback. Invalid
|
|
|
31
42
|
frontmatter, missing or empty divisions, duplicate names/slugs, unknown
|
|
32
43
|
agents, and unsupported platforms fail before output is written. A changed
|
|
33
44
|
source revision requires `--allow-source-drift` to refresh a Harness-owned
|
|
34
|
-
catalog.
|
|
35
|
-
|
|
45
|
+
catalog. Runtime artifacts are regenerated under the global workspace key;
|
|
46
|
+
authored document conflicts fail before migration.
|
|
36
47
|
|
|
37
48
|
## Handoff
|
|
38
49
|
|
|
39
50
|
The launcher and `handoff.json` require `status`, `changes`, `verification`,
|
|
40
|
-
`risks`, and `nextAction`. The
|
|
41
|
-
|
|
42
|
-
|
|
51
|
+
`risks`, and `nextAction`. The global manifest carries resolved paths, selected
|
|
52
|
+
roles, and execution metadata; the installed skill and indexer remain
|
|
53
|
+
immutable. Fable remains responsible for macro planning; TDD remains
|
|
54
|
+
responsible for ordinary feature implementation.
|
|
@@ -40,3 +40,36 @@ After resolving a recurring Windows path error, the agent summarizes the cause,
|
|
|
40
40
|
- [ ] No raw transcript, secret, or one-off detail was persisted.
|
|
41
41
|
- [ ] Rule/dynamic-skill classification and deduplication passed.
|
|
42
42
|
- [ ] Self-regression passed before the memory or manifest changed.
|
|
43
|
+
|
|
44
|
+
## Persistence boundary
|
|
45
|
+
|
|
46
|
+
The host supplies selected evidence. `self-evolve` receives only that brief,
|
|
47
|
+
then chooses a small rule or a reusable skill. It never scans transcript
|
|
48
|
+
stores on its own.
|
|
49
|
+
|
|
50
|
+
```mermaid
|
|
51
|
+
flowchart LR
|
|
52
|
+
Host[Host evidence and root cause] --> Classify{Reusable procedure?}
|
|
53
|
+
Classify -->|No| Rule[Persist a concise rule]
|
|
54
|
+
Classify -->|Yes| Draft[Load skill-creator and draft a skill]
|
|
55
|
+
Rule --> Gate[Deduplicate and quality-check]
|
|
56
|
+
Draft --> Gate
|
|
57
|
+
Gate -->|Pass| Record[Write the selected memory or manifest]
|
|
58
|
+
Gate -->|Fail| Stop[Leave state unchanged]
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Persistence outcomes
|
|
62
|
+
|
|
63
|
+
The decision is made from the lesson, not from transcript discovery. A long
|
|
64
|
+
existing memory is split into a topic file and reached through a pointer.
|
|
65
|
+
|
|
66
|
+
```mermaid
|
|
67
|
+
flowchart TD
|
|
68
|
+
Lesson[Resolved lesson] --> Existing{Matching workspace memory?}
|
|
69
|
+
Existing -->|Short enough| Append[Append the defensive rule]
|
|
70
|
+
Existing -->|Too long| Split[Create topic memory and pointer]
|
|
71
|
+
Existing -->|No| Fallback[Choose the workspace fallback]
|
|
72
|
+
Fallback --> Append
|
|
73
|
+
Append --> Verify[Run the relevant gate]
|
|
74
|
+
Split --> Verify
|
|
75
|
+
```
|
package/docs/workflows/tdd.md
CHANGED
|
@@ -40,3 +40,39 @@ graph TD
|
|
|
40
40
|
|
|
41
41
|
VerifyRefactor -->|Exit 0| End([Feature verified & completed])
|
|
42
42
|
```
|
|
43
|
+
|
|
44
|
+
## Profile boundary
|
|
45
|
+
|
|
46
|
+
The profile describes the boundary crossed by the behavior under test. The
|
|
47
|
+
RED/GREEN/REFACTOR loop remains the same; only the evidence and fixtures
|
|
48
|
+
change.
|
|
49
|
+
|
|
50
|
+
```mermaid
|
|
51
|
+
flowchart TD
|
|
52
|
+
Behavior[Behavior under test] --> Boundary{External process or service boundary?}
|
|
53
|
+
Boundary -->|No| Unit[Unit profile]
|
|
54
|
+
Boundary -->|Yes| Integration[Integration profile]
|
|
55
|
+
Unit --> Loop[RED -> GREEN -> REFACTOR]
|
|
56
|
+
Integration --> Loop
|
|
57
|
+
Loop --> Evidence[Run tests and score evidence]
|
|
58
|
+
Evidence -->|Gate fails| Loop
|
|
59
|
+
Evidence -->|All gates pass| Done([Verified behavior])
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Recovery state
|
|
63
|
+
|
|
64
|
+
The circuit breaker limits repeated edits. A third failed GREEN attempt moves
|
|
65
|
+
the work into `zoom-out`, where the assumptions are rebuilt before retrying.
|
|
66
|
+
|
|
67
|
+
```mermaid
|
|
68
|
+
stateDiagram-v2
|
|
69
|
+
[*] --> Red
|
|
70
|
+
Red --> GreenAttempt
|
|
71
|
+
GreenAttempt --> Green: tests pass
|
|
72
|
+
GreenAttempt --> Retry: tests fail
|
|
73
|
+
Retry --> GreenAttempt: attempts one or two
|
|
74
|
+
Retry --> ZoomOut: attempt three
|
|
75
|
+
ZoomOut --> Red: assumptions rebuilt
|
|
76
|
+
Green --> Refactor
|
|
77
|
+
Refactor --> [*]
|
|
78
|
+
```
|
|
@@ -32,3 +32,36 @@ graph TD
|
|
|
32
32
|
|
|
33
33
|
The repository has no TODO CLI state machine. This keeps progress portable,
|
|
34
34
|
host-native, and transparent across sessions.
|
|
35
|
+
|
|
36
|
+
## Blocker loop
|
|
37
|
+
|
|
38
|
+
One milestone owns the active work. A failed check updates that milestone and
|
|
39
|
+
returns to execution; it does not silently advance the rest of the list.
|
|
40
|
+
|
|
41
|
+
```mermaid
|
|
42
|
+
flowchart LR
|
|
43
|
+
Active[One milestone in progress] --> Execute[Execute bounded scope]
|
|
44
|
+
Execute --> Verify[Run named verification]
|
|
45
|
+
Verify -->|Pass| Record[Record evidence and complete]
|
|
46
|
+
Verify -->|Fail| Blocker[Record blocker and update plan]
|
|
47
|
+
Blocker --> Active
|
|
48
|
+
Record --> More{More milestones?}
|
|
49
|
+
More -->|Yes| Active
|
|
50
|
+
More -->|No| Done([Finish])
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Ownership state
|
|
54
|
+
|
|
55
|
+
The checklist makes ownership visible from planning through completion.
|
|
56
|
+
|
|
57
|
+
```mermaid
|
|
58
|
+
stateDiagram-v2
|
|
59
|
+
[*] --> Planned
|
|
60
|
+
Planned --> InProgress
|
|
61
|
+
InProgress --> Verifying
|
|
62
|
+
Verifying --> Complete: check passes
|
|
63
|
+
Verifying --> Blocked: check fails
|
|
64
|
+
Blocked --> InProgress: plan updated
|
|
65
|
+
Complete --> InProgress: milestones remain
|
|
66
|
+
Complete --> [*]: no milestones remain
|
|
67
|
+
```
|
|
@@ -4,7 +4,7 @@ description: Use at session start (Discover phase) to detect and align with the
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.4.0
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Environment Detection & Shell Alignment
|
|
@@ -25,11 +25,11 @@ Boundary: operate solely within `process.cwd()`; ignore other workspaces, histor
|
|
|
25
25
|
## Workflow
|
|
26
26
|
|
|
27
27
|
1. Preflight from this skill's own directory; parse OS, shell, available CLIs: `node "<this-skill-dir>/scripts/preflight.js"`
|
|
28
|
-
2. Self-heal harness touchpoints (idempotent): `node "<skills-repo-root>/harness-everything/scripts/self-heal.js"` — audits
|
|
28
|
+
2. Self-heal harness touchpoints (idempotent): `node "<skills-repo-root>/harness-everything/scripts/self-heal.js"` — audits `<workspace>/.claude/settings.json`, `<workspace>/.cursorrules`, `<workspace>/.github/copilot-instructions.md`, `<workspace>/AGENTS.md`.
|
|
29
29
|
3. Adopt syntax: Git Bash → forward `/`, `$VAR`, Unix cmds (`ls`, `rm -rf`), never `dir`/`del`/PowerShell · PowerShell → `$env:VAR`, cmdlets · CMD → `\`, `%VAR%`, `dir`/`del`/`copy`, never Unix.
|
|
30
30
|
4. On failure don't blindly retry — suspect wrong-shell syntax. Three failures → trigger `zoom-out`.
|
|
31
31
|
|
|
32
|
-
Deep dive: references/shell-syntax-rules.md
|
|
32
|
+
Deep dive: <this-skill-dir>/references/shell-syntax-rules.md
|
|
33
33
|
|
|
34
34
|
## USE FOR:
|
|
35
35
|
- session-start OS/shell/toolchain detection
|
package/eval-harness/SKILL.md
CHANGED
|
@@ -4,7 +4,7 @@ description: Evaluate AI agent performance across correctness, token efficiency,
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.4.0
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Eval Harness
|
|
@@ -17,15 +17,15 @@ Evaluate an agent log with a four-dimension scorecard.
|
|
|
17
17
|
| :--- | :--- |
|
|
18
18
|
| **Trigger / Input** | A benchmark request and an execution log or conversation history. |
|
|
19
19
|
| **Expected Output** | A 0–10 scorecard for correctness, efficiency, anti-loop focus, and environment awareness. |
|
|
20
|
-
| **State Mutations** | Writes the scorecard under
|
|
21
|
-
| **Enforcement Gate** | Run
|
|
20
|
+
| **State Mutations** | Writes the scorecard under `<workspace>/evals/` or `<workspace>/.github/harness-everything/evals/`. |
|
|
21
|
+
| **Enforcement Gate** | Run `<this-skill-dir>/scripts/evaluate.js`; use the documented fallback only if it fails or is missing. |
|
|
22
22
|
|
|
23
23
|
## Workflow
|
|
24
24
|
|
|
25
25
|
1. Parse actions, errors, tokens, and timing from the log.
|
|
26
|
-
2. Score all four dimensions using
|
|
27
|
-
3. Run `node scripts/evaluate.js <A> <B> <C> <D> "insights"`.
|
|
28
|
-
4. If the script fails, write
|
|
26
|
+
2. Score all four dimensions using `<this-skill-dir>/references/scoring-rubric.md`.
|
|
27
|
+
3. Run `node <this-skill-dir>/scripts/evaluate.js <A> <B> <C> <D> "insights"`.
|
|
28
|
+
4. If the script fails, write `<workspace>/evals/scorecard.md` (or the `<workspace>/.github` fallback).
|
|
29
29
|
|
|
30
30
|
## USE FOR:
|
|
31
31
|
|
|
@@ -39,4 +39,4 @@ Evaluate an agent log with a four-dimension scorecard.
|
|
|
39
39
|
- General review without scoring dimensions.
|
|
40
40
|
- Tasks without an execution log.
|
|
41
41
|
|
|
42
|
-
Deep dive:
|
|
42
|
+
Deep dive: `<this-skill-dir>/references/scoring-rubric.md`
|
|
@@ -1,6 +1,12 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
const fs = require('fs');
|
|
3
3
|
const path = require('path');
|
|
4
|
+
let getWorkspaceRoot;
|
|
5
|
+
try {
|
|
6
|
+
({ getWorkspaceRoot } = require('../../scripts/lib/workspace'));
|
|
7
|
+
} catch (err) {
|
|
8
|
+
getWorkspaceRoot = () => null;
|
|
9
|
+
}
|
|
4
10
|
|
|
5
11
|
const args = process.argv.slice(2);
|
|
6
12
|
|
|
@@ -50,17 +56,13 @@ for (const [dim, score] of Object.entries(scores)) {
|
|
|
50
56
|
markdown += `| **Total Score** | **${total}/40** |\n\n`;
|
|
51
57
|
markdown += `### Insights\n${insights}\n\n---\n`;
|
|
52
58
|
|
|
53
|
-
function getWorkspaceRoot() {
|
|
54
|
-
let dir = path.resolve(process.cwd());
|
|
55
|
-
while (dir !== path.parse(dir).root) {
|
|
56
|
-
if (fs.existsSync(path.join(dir, '.git'))) return dir;
|
|
57
|
-
dir = path.dirname(dir);
|
|
58
|
-
}
|
|
59
|
-
return process.cwd();
|
|
60
|
-
}
|
|
61
|
-
|
|
62
59
|
// Store the evaluations in an evals folder in the project root
|
|
63
|
-
const
|
|
60
|
+
const workspaceRoot = getWorkspaceRoot();
|
|
61
|
+
if (!workspaceRoot) {
|
|
62
|
+
console.error('[eval-harness] Cannot write an evaluation report without a resolved git workspace.');
|
|
63
|
+
process.exit(1);
|
|
64
|
+
}
|
|
65
|
+
const evalDir = path.join(workspaceRoot, 'evals');
|
|
64
66
|
if (!fs.existsSync(evalDir)) {
|
|
65
67
|
fs.mkdirSync(evalDir, { recursive: true });
|
|
66
68
|
}
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
## Harness Evaluation Report - 2026-08-31T14:29:33.829Z
|
|
2
|
+
|
|
3
|
+
| Dimension | Score |
|
|
4
|
+
|---|---|
|
|
5
|
+
| Correctness & Factuality | 10/10 |
|
|
6
|
+
| Token & Step Efficiency | 5/10 |
|
|
7
|
+
| Anti-loop & Focus | 10/10 |
|
|
8
|
+
| Environment & Tool Awareness | 10/10 |
|
|
9
|
+
| **Total Score** | **35/40** |
|
|
10
|
+
|
|
11
|
+
### Insights
|
|
12
|
+
Repository-grounded audit. Environment preflight passed. Core npm test and 9 mechanism suites passed after npm ci, but routing misclassified this macro evaluation request as Tier 1; waza was unavailable; behavioral validation has 3 invalid case files; existing 37 Harness-only live records are 26/37 pass and 9/15 pressure pass; the fresh Claude runner case was invalid because shell:true truncated the prompt to Add.; no causal benchmark case proved vanilla fail to Harness pass.
|
|
13
|
+
|
|
14
|
+
---
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-trivial-tier1-001
|
|
2
2
|
name: Trivial edit routes to Tier 1
|
|
3
|
-
description: "
|
|
3
|
+
description: "Route software/project work through the Harness kernel: classify Tier 1/2/3, establish mandatory invariants, then let the agent choose tactics and domain skills. Use for software triage, routing, and re-routing; not general Q&A or non-software writing."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
- tier-1
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-standard-tier2-001
|
|
2
2
|
name: Standard bug fix routes to Tier 2
|
|
3
|
-
description: "
|
|
3
|
+
description: "Route software/project work through the Harness kernel: classify Tier 1/2/3, establish mandatory invariants, then let the agent choose tactics and domain skills. Use for software triage, routing, and re-routing; not general Q&A or non-software writing."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
- tier-2
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-macro-tier3-001
|
|
2
2
|
name: Architectural task routes to Tier 3
|
|
3
|
-
description: "
|
|
3
|
+
description: "Route software/project work through the Harness kernel: classify Tier 1/2/3, establish mandatory invariants, then let the agent choose tactics and domain skills. Use for software triage, routing, and re-routing; not general Q&A or non-software writing."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
- tier-3
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-install-cognitive-os-001
|
|
2
2
|
name: On-scope prompt triggers install-cognitive-os
|
|
3
|
-
description: "
|
|
3
|
+
description: "Explain or explicitly apply the Harness cognitive policy: Discover, Think, Try, Summarize, Record. It provides cross-cutting behavior without requiring this skill to be selected before domain skills."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
inputs:
|
|
@@ -4,7 +4,7 @@ description: Enforce fable-mode's context discipline as a shadow guard. Use when
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.4.0
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Fable Discipline (Macro Task Discipline & Safety Net)
|
|
@@ -38,4 +38,4 @@ Shadow guard for `fable-mode`: MUST run in the background for its whole duration
|
|
|
38
38
|
3. **Agent handoff**: Outgoing agents MUST leave a state manifest (completed APIs, expected inputs/outputs); incoming agents MUST verify it first.
|
|
39
39
|
4. **Circuit breaker**: If build errors diverge, HALT, call `zoom-out`, and map the error dependency graph for the human.
|
|
40
40
|
|
|
41
|
-
Deep dive: references/discipline-rules.md
|
|
41
|
+
Deep dive: <this-skill-dir>/references/discipline-rules.md
|