harness-everything 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (102) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/.claude-plugin/plugin.json +1 -1
  3. package/.github/workflows/ci.yml +12 -2
  4. package/AGENTS.md +36 -35
  5. package/CHANGELOG.md +51 -0
  6. package/README.md +2 -2
  7. package/benchmarks/results/reviews/pr142-verification.json +95 -0
  8. package/ci/action-gate.test.js +83 -3
  9. package/ci/doc-capability-consistency.test.js +3 -0
  10. package/ci/invariant-routing.test.js +87 -74
  11. package/ci/mechanism-2af-worktree-isolation.test.js +152 -0
  12. package/ci/mechanism-2ag-router-worktree-contract.test.js +44 -0
  13. package/ci/mechanism-2ah-workflow-lifecycle.test.js +140 -0
  14. package/ci/openai-plugin-package.test.js +9 -6
  15. package/ci/phase4-ensemble-memory.test.js +128 -10
  16. package/ci/router-workflow-plan.test.js +23 -0
  17. package/ci/workflow-execution-contract.test.js +133 -0
  18. package/ci/workflow-plan-consumers.test.js +18 -0
  19. package/docs/architecture.md +111 -114
  20. package/docs/mechanism-first-skill-mesh.md +92 -103
  21. package/docs/openai-plugin.md +1 -1
  22. package/docs/philosophy.md +40 -33
  23. package/docs/platform-capabilities.md +2 -2
  24. package/docs/reviews/pr-142-issue-131.md +89 -0
  25. package/docs/routing.md +61 -124
  26. package/docs/workflow-runtime.md +109 -0
  27. package/fable-mode/CONTRACT-FORMAT.md +3 -1
  28. package/fable-mode/SKILL.md +1 -1
  29. package/fable-mode/agents/fable-orchestrator.md +9 -7
  30. package/fable-mode/execution-guardrails/SKILL.md +1 -1
  31. package/fable-mode/fable-haiku/SKILL.md +1 -1
  32. package/fable-mode/fable-opus/SKILL.md +1 -1
  33. package/fable-mode/fable-sonnet/SKILL.md +1 -1
  34. package/fable-mode/scripts/workflow-plan-consumer.js +13 -8
  35. package/harness-everything/SKILL.md +11 -10
  36. package/harness-everything/references/triage-and-tiers.md +76 -107
  37. package/harness-everything/schemas/router-task-shape.schema.json +4 -2
  38. package/harness-everything/schemas/router-workflow-plan.schema.json +3 -2
  39. package/harness-everything/scripts/kernel-router-core.js +265 -0
  40. package/harness-everything/scripts/kernel-router.js +118 -223
  41. package/harness-everything/scripts/router-contract-core.js +633 -0
  42. package/harness-everything/scripts/router-contract.js +21 -581
  43. package/harness-everything/scripts/tier-router.js +19 -1
  44. package/hooks/hooks.json +25 -4
  45. package/hooks/scripts/action-gate.js +111 -2
  46. package/hooks/scripts/contract-test.js +27 -2
  47. package/hooks/scripts/lib/workflow-isolation.js +102 -0
  48. package/hooks/scripts/lib/workflow-runtime.js +254 -0
  49. package/hooks/scripts/state-persist.js +1 -0
  50. package/hooks/scripts/subagent-scope-guard.js +20 -3
  51. package/hooks/scripts/workflow-disposition.js +104 -0
  52. package/hooks/scripts/workflow-gate.js +56 -0
  53. package/hooks/scripts/workflow-stop-gate.js +44 -0
  54. package/multi-agent-workspace/SKILL.md +2 -2
  55. package/multi-agent-workspace/scripts/index_memory.js +107 -1
  56. package/opencode-plugin/plugin.json +1 -1
  57. package/package.json +2 -2
  58. package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
  59. package/plugins/harness-everything/hooks/hooks.json +44 -1
  60. package/plugins/harness-everything/hooks/scripts/action-gate.js +111 -2
  61. package/plugins/harness-everything/hooks/scripts/contract-test.js +27 -2
  62. package/plugins/harness-everything/hooks/scripts/lib/workflow-isolation.js +102 -0
  63. package/plugins/harness-everything/hooks/scripts/lib/workflow-runtime.js +254 -0
  64. package/plugins/harness-everything/hooks/scripts/state-persist.js +1 -0
  65. package/plugins/harness-everything/hooks/scripts/subagent-scope-guard.js +20 -3
  66. package/plugins/harness-everything/hooks/scripts/workflow-disposition.js +104 -0
  67. package/plugins/harness-everything/hooks/scripts/workflow-gate.js +56 -0
  68. package/plugins/harness-everything/hooks/scripts/workflow-stop-gate.js +44 -0
  69. package/plugins/harness-everything/plugin.json +1 -1
  70. package/plugins/harness-everything/scripts/lib/advisory-text.js +51 -54
  71. package/plugins/harness-everything/skills/fable-mode/CONTRACT-FORMAT.md +3 -1
  72. package/plugins/harness-everything/skills/fable-mode/SKILL.md +1 -1
  73. package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +9 -7
  74. package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +1 -1
  75. package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +1 -1
  76. package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +1 -1
  77. package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +1 -1
  78. package/plugins/harness-everything/skills/fable-mode/scripts/workflow-plan-consumer.js +13 -8
  79. package/plugins/harness-everything/skills/harness-everything/SKILL.md +11 -10
  80. package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +76 -107
  81. package/plugins/harness-everything/skills/harness-everything/schemas/router-task-shape.schema.json +4 -2
  82. package/plugins/harness-everything/skills/harness-everything/schemas/router-workflow-plan.schema.json +3 -2
  83. package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router-core.js +265 -0
  84. package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +118 -223
  85. package/plugins/harness-everything/skills/harness-everything/scripts/router-contract-core.js +633 -0
  86. package/plugins/harness-everything/skills/harness-everything/scripts/router-contract.js +21 -581
  87. package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +19 -1
  88. package/plugins/harness-everything/skills/multi-agent-workspace/SKILL.md +2 -2
  89. package/plugins/harness-everything/skills/multi-agent-workspace/scripts/index_memory.js +107 -1
  90. package/plugins/harness-everything/skills/self-evolve/SKILL.md +6 -6
  91. package/plugins/harness-everything/skills/self-evolve/references/memory-resolution.md +22 -33
  92. package/plugins/harness-everything/skills/self-evolve/scripts/persist-memory.js +336 -57
  93. package/plugins/harness-everything/skills/self-evolve/scripts/self-regression.js +1 -0
  94. package/plugins/harness-everything/skills/using-git-worktrees/SKILL.md +14 -16
  95. package/plugins/harness-everything/skills/using-git-worktrees/references/workflow-details.md +44 -85
  96. package/scripts/lib/advisory-text.js +51 -54
  97. package/self-evolve/SKILL.md +6 -6
  98. package/self-evolve/references/memory-resolution.md +22 -33
  99. package/self-evolve/scripts/persist-memory.js +336 -57
  100. package/self-evolve/scripts/self-regression.js +1 -0
  101. package/using-git-worktrees/SKILL.md +14 -16
  102. package/using-git-worktrees/references/workflow-details.md +44 -85
@@ -28,7 +28,7 @@ flowchart TD
28
28
  X[Stop hook] --> V
29
29
  ```
30
30
 
31
- The local OpenAI adapter enforces cross-cutting invariants through the lifecycle events that it packages. Tier-specific skills remain advisory; the model may choose, combine, reorder, or skip them. Tool enforcement is limited to the host tool names declared in `hooks/hooks.json`.
31
+ The local OpenAI adapter enforces cross-cutting invariants through the lifecycle events that it packages. Suggested skills require applicability evaluation; the selected topology is mandatory with scoped, evidence-backed escape. The [workflow runtime](workflow-runtime.md) documents pending/running/satisfied/blocked state, Fable entry, worktree isolation, completion evidence, and degradation limits. Tool enforcement is limited to the host tool names declared in `hooks/hooks.json`.
32
32
 
33
33
  ## Package layout
34
34
 
@@ -1,61 +1,68 @@
1
1
  # Harness Philosophy
2
2
 
3
- Harness is built on a single, core belief: **AI coding models are highly capable, but they struggle with self-regulation, attention drift, and environment awareness.**
3
+ Harness is built on a simple observation: **strong AI coding models reason well, but can still self-authorize away process obligations when they believe the task is already clear.**
4
4
 
5
- Instead of replacing the model or restricting it with heavy, opinionated frameworks that stifle creativity, Harness acts as a **behavioral layer** — reusable skills plus the strongest lightweight runtime mechanisms each supported installation surface actually provides.
5
+ Harness therefore separates reasoning freedom from workflow reliability. It does not prescribe the model's chain of thought or force one giant pipeline. Instead, it establishes the smallest applicable lifecycle contract and lets the model choose how to satisfy it.
6
6
 
7
- The current mechanism/enforcement boundary is platform-specific; see [Platform Capability Matrix](platform-capabilities.md). A skill-only or instruction-only install must not be described as if it automatically runs lifecycle hooks.
7
+ The mechanism/enforcement boundary remains platform-specific; see [Platform Capability Matrix](platform-capabilities.md). A semantic contract can be mandatory even when an instruction-only host cannot mechanically block every violation, so contract strength and host enforcement evidence must be reported separately.
8
8
 
9
- ---
9
+ ## Core Philosophy: Constrain Lifecycle, Not Reasoning
10
10
 
11
- ## The Core Philosophy: Intervene Only When Necessary
11
+ 1. **Reasoning autonomy remains high.** The model chooses implementation technique, tools, decomposition details, and tactics inside the selected workflow.
12
+ 2. **Applicable workflow is mandatory.** Once routing selects a topology, it must be executed to resolution. Model confidence, “simple”, “routine”, and “already clear” are not escape conditions.
13
+ 3. **Skill applicability is explicit.** Router-suggested skills are read/evaluated before omission. A suggestion can be `not-applicable` when its actual flow does not match; a selected topology is stronger and cannot simply be skipped.
14
+ 4. **Escape is conditional, not default.** If the selected workflow genuinely cannot represent part of the task, record the uncovered scope and evidence, then allow model-defined handling only for that uncovered portion.
15
+ 5. **Mechanisms beat stronger wording.** Important lifecycle transitions should use structured state, objective evidence, hooks/plugins, exit codes, and completion gates where the host supports them rather than escalating `MUST`/`CRITICAL` prose.
12
16
 
13
- Many AI development frameworks attempt to fully control the model's execution flow. They use rigid DAGs, forced chains, or complex state machines. While this works for trivial, repetitive tasks, it can suppress useful model judgment on complex, novel engineering work.
17
+ This principle can be summarized as:
14
18
 
15
- Harness takes a different approach: preserve execution autonomy, but make important decisions informed rather than superficial.
19
+ > **Mandatory applicable workflow; flexible reasoning/implementation inside it; evidence-backed escape only where the workflow does not cover the task.**
16
20
 
17
- 1. **Model Autonomy is Paramount:** The model should have freedom to explore, choose useful skills/tools, and design its solution inside a small set of cross-cutting invariants.
18
- 2. **Mandatory Evaluation, Advisory Execution:** When the router suggests a skill, the model must read/evaluate that skill's actual `SKILL.md` entry/basic flow before omitting it. Execution is still optional after evaluation. This prevents “advisory” from degenerating into “always ignored” without creating a fixed pipeline.
19
- 3. **Context-Driven Guidance:** Harness uses high-context, low-friction signals — skills, rules, local checks, and hooks/plugins where the host supports and packages them — rather than one rigid global execution path.
20
- 4. **Reactive Guardrails:** On integration surfaces that expose the required lifecycle/tool hooks, Harness can stay quiet until a boundary is violated or a repeated failure is detected. On advisory surfaces, the same principles remain guidance rather than automatic blocking.
21
- 5. **Behavior-First, Not Prompt-First:** Harness is not a collection of magic prompts. Its skills remain independently useful, while supported runtime adapters can react to real tool/session outcomes and workspace conditions.
21
+ The goal is not to make the model less capable. It is to prevent capability/confidence from silently deleting engineering obligations such as verification, bounded re-planning, stage checks, documentation checks, or safety gates when those obligations are applicable.
22
22
 
23
- The read-before-skip rule is intentionally narrow. It does not require every optional reference linked by a skill to be loaded, and it does not mean every suggestion must run. The agent reads the complete skill entry, evaluates `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules, follows any explicitly required applicability reference, and then decides whether execution adds value.
23
+ ## Why This Is Not a Rigid Global Pipeline
24
24
 
25
- ---
25
+ Harness still rejects a universal `TODO → TDD → Fable → verification` sequence.
26
26
 
27
- ## The 4 Pillars of AI Model Guidance
28
-
29
- To keep agents aligned and productive, Harness operates across four distinct domains. The **principle** applies everywhere; whether it is injected or enforced automatically depends on the selected host surface.
27
+ The router chooses the **smallest sufficient topology**:
30
28
 
31
- ### 1. Environment Alignment (Discovery over Assumption)
29
+ - `direct-single` for bounded one-pass work;
30
+ - `iterative-single` for ordinary bounded reason/act work with objective verification;
31
+ - `fable-staged` for dependent multi-stage work;
32
+ - `fable-parallel` only for validated independent workstreams;
33
+ - `fable-multi-agent-workspace` when persistent roles/handoffs/memory are materially needed.
32
34
 
33
- AI models often hallucinate terminal environments or make incorrect assumptions about the host operating system, shell, or package manager — especially on Windows or mixed-shell setups.
35
+ Domain skills remain conditional on applicability. The contract says “execute the selected lifecycle”, not “execute every skill in the repository.”
34
36
 
35
- Harness requires environment assumptions to be discovered rather than guessed. On surfaces that package a session-start/preflight hook, that context can be injected automatically. On instruction/skill-only paths, the agent must discover it explicitly before relying on shell-specific commands.
37
+ ## The 4 Pillars of AI Model Guidance
36
38
 
37
- ### 2. Guardrails (The Circuit Breakers)
39
+ ### 1. Environment Alignment — Discovery over Assumption
38
40
 
39
- When a model gets stuck on a subtle bug or compiler error, its natural tendency is to make micro-adjustments repeatedly.
41
+ Discover OS, shell, package/runtime context, repository boundaries, and available host capabilities before relying on them. On surfaces that package preflight/session hooks this can be injected automatically; instruction-only hosts must do it explicitly.
40
42
 
41
- The Harness recovery invariant is to stop same-signature micro-retries and re-plan after repeated failure. On hosts where the Rule-of-3 mechanism is actually packaged, a circuit breaker can track failures and block further mutation until a `zoom-out` diagnosis is performed. On other surfaces, `zoom-out` remains a reusable recovery discipline but there is no claim that an automatic counter/blocker exists.
43
+ ### 2. Guardrails — Lifecycle Boundaries
42
44
 
43
- ### 3. Context Preservation (Anti-Bloat Protection)
45
+ Circuit breakers, action gates, workflow gates, scope guards, and completion gates protect specific transitions. They should remain narrow and evidence-driven. On integration surfaces that expose the required lifecycle/tool hooks, Harness can mechanically block supported violations; elsewhere the same contract is instruction-governed and must not be mislabeled hard enforcement.
44
46
 
45
- As sessions grow, models can read too broadly or produce oversized output, degrading active context.
47
+ ### 3. Context Preservation — Progressive Disclosure
46
48
 
47
- Harness uses narrow reads, explicit scope, compact evidence, and — where a compatible runtime adapter exists — boundary mechanisms that can react mechanically. Read-before-skip is bounded to the suggested skill entry and any applicability-critical material that entry explicitly requires; it does not justify indiscriminately loading the whole skills tree. Skill-only surfaces still receive the discipline, but not an imaginary background daemon or hook.
49
+ Load the smallest useful context. Read a suggested skill's complete entry before deciding applicability, but do not load the entire skill tree or every optional deep reference. Machine-visible workflow state should carry lifecycle facts instead of repeating large prompt blocks.
48
50
 
49
- ### 4. Self-Evolution (Continuous Workspace Memory)
51
+ ### 4. Self-Evolution — Learn from Workflow Evidence
50
52
 
51
- When a complex issue is resolved, Harness can turn the verified lesson into durable workspace knowledge through `self-evolve`: a concise rule or a reusable generated skill, when persistence is actually justified.
53
+ `self-evolve` should learn from verified recovery and workflow gaps: escape events, repeated replans, verifier fail→pass transitions, and recurrence outcomes. It should improve future coverage rather than granting a shortcut around the current workflow. Durable learning remains distinct from runtime session state.
52
54
 
53
- This long-term learning path is distinct from **runtime session state** such as WAL/checkpoint files. Runtime state exists only on integrations that package the corresponding state hooks; reusable rules/skills are project artifacts and can outlive a single host session.
55
+ ## Mechanism-First Cooperation
54
56
 
55
- ---
57
+ Harness prefers a compact control loop:
56
58
 
57
- ## A Non-Intrusive Cognitive Amplifier
59
+ ```text
60
+ route -> select workflow -> execute -> verify
61
+ | fail -> bounded re-plan
62
+ | uncovered -> evidence-backed escape
63
+ | pass -> complete
64
+ ```
58
65
 
59
- Harness does not require a heavy daemon or a proprietary service. The repository's mechanisms and skill content operate locally in the developer/host environment, while each platform gets only the enforcement strength its real integration supports.
66
+ This leaves creativity where stronger models benefit from it while making lifecycle obligations observable and testable.
60
67
 
61
- For skill-level coordination, see [Mechanism-First Skill Mesh](mechanism-first-skill-mesh.md). For the exact current platform/install/evidence boundary, see [Platform Capability Matrix](platform-capabilities.md).
68
+ For skill-level coordination, see [Mechanism-First Skill Mesh](mechanism-first-skill-mesh.md). For current installation/enforcement/evidence status, see [Platform Capability Matrix](platform-capabilities.md).
@@ -19,9 +19,9 @@ All `liveHostVerification` rows except OpenCode are `Unknown`. OpenCode's row is
19
19
 
20
20
  ### Routing contract vs. host evidence
21
21
 
22
- The repository routing contract now uses **mandatory evaluation / advisory execution** for skill suggestions: when the router suggests a skill, the agent is instructed to read that skill's complete `SKILL.md` entry/basic flow before omission. A name, description, router summary, tier label, or generic “routine task” judgement is not enough skip evidence; unreadable suggestions are `unresolved/unavailable`, not silent skips.
22
+ The repository routing contract uses **mandatory applicable workflow / conditional scoped escape**. Each suggested skill still requires reading its complete `SKILL.md` entry/basic flow before omission; the selected topology must resolve its applicable lifecycle obligations. A name, description, router summary, tier label, or generic “routine task” judgement is not enough skip evidence; unreadable suggestions are `unresolved/unavailable`, not silent skips.
23
23
 
24
- This is a **repository/agent contract**, not an automatic upgrade to any platform row below. A hook can prove that the contract was injected into a session; it does not prove the model actually read every suggested skill. Instruction-only surfaces can carry the same rule without mechanically enforcing it. Promoting this behavior to live-host compliance requires retained host/session evidence under the #82 compatibility program. Because this change does not add or remove a host capability, it does not change `platform-compatibility.json` status values by itself.
24
+ This is a **repository/agent contract**, not an automatic upgrade to any platform row below. A hook can prove that the contract was injected into a session; it does not prove the model actually read every suggested skill. Instruction-only surfaces can carry the same rule without mechanically enforcing it. Promoting this behavior to live-host compliance requires retained host/session evidence under the #82 compatibility program. The Claude and local OpenAI packages now carry workflow entry/completion gates, but this package/mechanism change does not upgrade `platform-compatibility.json` live-host status values. See [workflow runtime](workflow-runtime.md) for scope, worktree isolation, and failure behavior. OpenCode and instruction-only/public Skills-only surfaces do not acquire equivalent runtime gating. Shell inspection is not a filesystem sandbox; absent hooks, agent-writable state, and indirect script effects remain limits.
25
25
 
26
26
  ## Current capability summary
27
27
 
@@ -0,0 +1,89 @@
1
+ # PR #142 / issue #131 completion review
2
+
3
+ This review covers the implementation in the commit containing this report,
4
+ starting from PR head `8b99baec44745e171f078fe9eb3dac6fd812b5c8`. It evaluates
5
+ Harness engineering and the latest #131 decision, including mandatory mutation
6
+ isolation. It is not a measured agent-performance benchmark.
7
+
8
+ ## Findings resolved
9
+
10
+ | Original gap | Delivered behavior | Regression evidence |
11
+ | --- | --- | --- |
12
+ | Escape changed the whole workflow state and skipped isolation | Stage-scoped escape; isolation survives legacy escape and blocked states | `mechanism-2af-worktree-isolation.test.js`, `mechanism-2ah-workflow-lifecycle.test.js` |
13
+ | Only cwd was checked | Registered linked checkout of the bound repo; direct/patch targets, move destinations, traversal, and links checked | `mechanism-2af-worktree-isolation.test.js` |
14
+ | Windows runner used 8.3 paths while Git reported long names | Native physical-path resolution equates both spellings without relaxing isolation | Windows path regression and worktree suites |
15
+ | Read-only shell prefixes accepted writes | Reject redirects, substitutions, chained commands and dangerous options; require isolated cwd for mutation/unknown commands | `mechanism-2af-worktree-isolation.test.js` |
16
+ | Next prompt erased unresolved obligations | Stable workflow identity across steering prompts; stronger route requires replan | `mechanism-2ah-workflow-lifecycle.test.js` |
17
+ | Same strategy/timestamp loosely matched any Fable run | Exact workflow, session and run correlation | `mechanism-2ah-workflow-lifecycle.test.js` |
18
+ | Entry gate blocked its own bootstrap path | One bounded stage-specification file and trusted start controller; state/source writes remain gated | `mechanism-2ah-workflow-lifecycle.test.js` |
19
+ | Stop trusted directory contents and pass labels | Expected manifest stages, correlated exit-zero evidence, worker attribution and fresh verifier required | `mechanism-2ah-workflow-lifecycle.test.js` |
20
+ | Stop retry silently passed unresolved work | Retry records blocked; no satisfied state without required evidence | `mechanism-2ah-workflow-lifecycle.test.js` |
21
+ | Failures could not recover cleanly | Observed fail-to-pass checks; explicit bounded replan; retained prior run evidence | `mechanism-2ah-workflow-lifecycle.test.js` |
22
+ | Runtime policy scraped human-readable output | Kernel imports the structured core result; complete invariants persist with the plan | routing tests and lifecycle tests |
23
+ | Current documents contradicted the PR | README, platform matrix, routing, architecture, Fable/worktree references and mirrors aligned | consistency/capability/reference gates |
24
+ | Existing CI selected the first matching hook group | Action-gate regression finds its actual adapter among shared tool matchers | `ci/action-gate.test.js` |
25
+
26
+ The first added isolation regression run failed 16 assertions against the PR
27
+ baseline, then passed after repair. The baseline's four failing GitHub CI jobs
28
+ also reported the action-gate matcher defect. The initial local checkout lacked
29
+ dependencies, producing a separate CRLF-fixture setup failure; `npm ci` resolved
30
+ that environment prerequisite.
31
+
32
+ ## Verification
33
+
34
+ Local environment: Windows, PowerShell, Node.js 24.15.0, npm 11.12.1. Work was
35
+ performed in an isolated linked worktree; the primary checkout remained clean.
36
+
37
+ All 14 required gate categories passed: `npm test`, mutations, mechanism,
38
+ consistency, repository contract, capability docs, references, release,
39
+ collision, positive skill routing, routing invariants, OpenAI package/runtime,
40
+ public submission, and platform compatibility. Failed checks were corrected and
41
+ the affected gates rerun. The final `npm test` includes **44 mechanism suites**.
42
+
43
+ The new lifecycle suite exercises **50 assertions against source and the same
44
+ 50 against the packaged OpenAI layout**, using actual child processes, Git
45
+ repositories, worktrees, controller commands and hook payloads. Host payloads
46
+ are fixtures: these are deterministic mechanism tests, not live agent sessions.
47
+
48
+ Waza passed 30 canonical/nested skill readiness checks, 26 routing-spec checks,
49
+ and token validation (57 successful commands). Positive routing covered 34
50
+ cases across 26 direct skills and classified four nested skills.
51
+
52
+ Build/types/lint have no separate project scripts; repository syntax and
53
+ consistency gates apply. No coverage percentage is claimed. The security scanner
54
+ reported 12 pre-existing documentation/test-pattern matches, including its own
55
+ patterns and an explicitly fake token fixture; changed production code added no
56
+ scanner findings. `npm ci` reported zero dependency vulnerabilities. Diff review
57
+ and `git diff --check` passed. See the retained [gate summary](../../benchmarks/results/reviews/pr142-verification.json).
58
+
59
+ Remote CI is a separate evidence layer. Its result must be read at the final PR
60
+ head; this local record does not assert a future GitHub Actions outcome.
61
+
62
+ ## Reassessment
63
+
64
+ Scores are review judgments about engineering readiness, each out of 100.
65
+ Each row has four equally weighted criteria (25 points each), with the observed
66
+ deductions shown. They are not success probabilities or benchmark results.
67
+
68
+ | Dimension | Criteria scores | Total | Remaining deductions |
69
+ | --- | --- | ---: | --- |
70
+ | Contract coherence | entry 22, transitions 23, scoped exceptions 24, docs 23 | 92 | Skill applicability and iterative budgets still partly rely on executor compliance |
71
+ | Mutation safety | worktree identity 23, targets 23, shell handling 17, state trust 12 | 75 | Arbitrary script effects need a host sandbox; runtime state is agent-writable |
72
+ | Deterministic verification | gate breadth 25, negative cases 24, package parity 24, evidence checks 18 | 91 | Host-shaped fixtures cannot establish real host behavior or semantic check quality |
73
+ | Operation and maintenance | entry/recovery 22, modularity 21, distribution 23, diagnostics/migration 18 | 84 | Manual stage setup, missing-metadata blocks and legacy-state repair add operator work |
74
+
75
+ Equal-weight engineering score: **85.5/100** (approximately **86/100**).
76
+ The system now has substantially stronger tested lifecycle and isolation
77
+ contracts. It is suitable for further host validation with explicit boundaries.
78
+
79
+ **Live-host reliability of these new gates remains unverified. Behavioral gain
80
+ from this PR remains unmeasured.** Neither receives an invented effectiveness
81
+ score. Older retained OpenCode evidence and older behavioral cases cannot prove
82
+ this workflow change. This review does not claim superiority over other skill
83
+ collections, reduced token cost, or improved task success rates.
84
+
85
+ The next evidence needed is a retained Claude/Codex session showing routing,
86
+ entry, mutation denial/allow, numeric stage results, worker correlation and Stop
87
+ resolution; then paired same-model/same-task behavioral runs. OpenCode needs its
88
+ own adapter before parity can be claimed. See [runtime limits](../workflow-runtime.md)
89
+ and [platform capabilities](../platform-capabilities.md).
package/docs/routing.md CHANGED
@@ -1,28 +1,26 @@
1
1
  # Harness Routing & Task Triage
2
2
 
3
- Harness classifies software work into three tiers to prevent both over-engineering and under-planning. **The tier is a scope recommendation, not a workflow pipeline.**
3
+ Harness classifies software work and selects the smallest sufficient execution topology. **A tier is a scope/risk signal, not a fixed skill sequence. A selected topology is an execution contract, not an optional suggestion.**
4
4
 
5
5
  The current architecture follows two coupled rules:
6
6
 
7
- > **Do not enforce workflow order. Enforce workflow invariants.**
7
+ > **Mandatory applicable workflow; flexible reasoning/implementation inside it.**
8
8
  >
9
- > **Suggested skills are mandatory to evaluate and advisory to execute.**
9
+ > **Skill applicability is evaluated explicitly; workflow escape is exception-only and evidence-backed.**
10
10
 
11
- A strong model should remain free to decide how to complete the task, but it may not discard a router suggestion without first reading the suggested skill's actual entry/basic flow.
11
+ A strong model remains free to decide *how* to perform the work. It may not decide that an applicable selected lifecycle can be skipped because the task feels simple or already understood.
12
12
 
13
13
  ## Kernel invariants
14
14
 
15
- For software/project work, Harness establishes three baseline mandatory rails before mutation:
15
+ For software/project work, Harness establishes mandatory rails before mutation:
16
16
 
17
- 1. **Route before execution** — establish task scope/tier.
17
+ 1. **Route before execution** — establish scope/tier and the smallest justified topology.
18
18
  2. **Verify before claim** — completion requires objective evidence appropriate to the change.
19
19
  3. **Re-plan after repeated failure** — after three same-signature failures, stop micro-retrying and use a fresh diagnosis / `zoom-out`.
20
+ 4. **Evaluate before omission** — read every suggested skill's complete `SKILL.md` entry before deciding applicability.
21
+ 5. **Resolve selected workflow** - successful completion requires `satisfied`; `blocked` is a reportable incomplete outcome. Escape covers only a declared stage and preserves other obligations.
20
22
 
21
- When the router emits one or more skill suggestions, it adds a fourth conditional invariant:
22
-
23
- 4. **Evaluate before skip** — before omitting any suggested skill, read its complete `SKILL.md` entry and evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules against the task.
24
-
25
- Execution, ordering, planning style, and delegation remain agent decisions after evaluation unless the user or an authoritative project contract explicitly requires them.
23
+ The model owns tools, implementation technique, reasoning, and decomposition details inside those rails.
26
24
 
27
25
  ## Public routing path
28
26
 
@@ -30,10 +28,10 @@ Execution, ordering, planning style, and delegation remain agent decisions after
30
28
  kernel-router.js
31
29
  ├─ delegates classification / guide discovery → tier-router.js
32
30
  ├─ preserves tier + rationale + dynamic-skill recommendations
33
- ├─ suppresses legacy fixed-pipeline wording
34
- ├─ emits required invariants
35
- ├─ requires evaluation of each suggested skill before omission
36
- └─ leaves suggested-skill execution advisory after evaluation
31
+ ├─ emits required invariants + structured workflow plan
32
+ ├─ requires applicability evaluation for suggested skills
33
+ ├─ activates the selected workflow execution contract
34
+ └─ exposes explicit blocked/degraded/escape state instead of silent downgrade
37
35
  ```
38
36
 
39
37
  Use:
@@ -44,32 +42,29 @@ node harness-everything/scripts/kernel-router.js "<prompt>"
44
42
  npx github:dyphn1/Harness-everything next "<prompt>"
45
43
  ```
46
44
 
47
- On a host where `UserPromptSubmit` is wired, reuse the hook output instead of running it twice.
45
+ On a host where `UserPromptSubmit` is wired, reuse that hook output instead of running it twice.
48
46
 
49
- ## Suggestion evaluation contract
47
+ ## Suggested skills vs. selected workflow
50
48
 
51
- A router suggestion is not permission to make a decision from metadata alone. For **every** suggested skill:
49
+ These are intentionally different concepts.
52
50
 
53
- 1. Resolve and read the complete `SKILL.md` entry.
54
- 2. Evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules against the current task.
55
- 3. If the entry explicitly requires another document to determine applicability, read that required material too. Ordinary deep-dive/reference links remain optional unless the entry makes them decision-critical.
56
- 4. Choose `use`, `skip`, or `unresolved/unavailable` only after that evaluation.
51
+ For **every suggested skill**:
57
52
 
58
- The following are not sufficient skip evidence by themselves:
53
+ 1. Resolve/read the complete `SKILL.md` entry.
54
+ 2. Evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules.
55
+ 3. Read extra material only when the entry explicitly requires it for applicability.
56
+ 4. Resolve the suggestion as `use`, `not-applicable`, or `unresolved/unavailable`.
59
57
 
60
- - the skill name,
61
- - the frontmatter description,
62
- - the router's one-line summary,
63
- - the tier label,
64
- - a generic judgement such as “routine/common task.”
58
+ A name, description, router summary, tier label, or generic “routine/common task” judgement is not sufficient evidence for `not-applicable`.
65
59
 
66
- Using one suggested skill does not waive read-before-skip for the other suggestions. If a suggested skill cannot be resolved or read, mark it `unresolved/unavailable` rather than silently treating it as inapplicable. If every suggestion is skipped after evaluation, give one brief flow-grounded reason in the visible routing checkpoint/progress update.
60
+ For the **selected workflow topology**:
67
61
 
68
- ## Non-software bypass
62
+ - enter it and execute it to resolution;
63
+ - do not silently replace it with a weaker/direct path because the model is confident;
64
+ - preserve objective checks, stage contracts, synthesis barriers, re-plan limits, and other obligations selected by the plan;
65
+ - use escape only for genuinely uncovered workflow scope, recording reason + uncovered scope + evidence.
69
66
 
70
- General Q&A, translation, ordinary web search, and non-software writing bypass Harness routing. The model should answer directly and naturally.
71
-
72
- A software prompt that already says "use TDD", "security review", `repo-docs`, or another skill **does not bypass the kernel**. Host skill routing and Harness routing are different layers: the kernel establishes cross-cutting invariants; the host/model selects useful expertise after evaluating router suggestions.
67
+ This prevents two opposite failures: forcing every task through one giant pipeline, and allowing a model to rationalize away the entire engineering lifecycle.
73
68
 
74
69
  ## Three-tier recommendation model
75
70
 
@@ -77,123 +72,65 @@ A software prompt that already says "use TDD", "security review", `repo-docs`, o
77
72
  flowchart TD
78
73
  U([User Request]) --> S{Software / project work?}
79
74
  S -- No --> B[Bypass Harness]
80
- S -- Yes --> K[Harness Kernel<br/>scope + invariants]
81
- K --> T{Tier recommendation}
82
- T --> G{Suggested skills?}
83
- G -- Yes --> R[Read each suggested SKILL.md<br/>evaluate flow + applicability]
84
- G -- No --> A[Agent self-orchestrates]
85
- R --> A
86
- A --> E[Execute selected tactics / skills]
87
- E --> V{Evidence supports completion?}
88
- V -- Yes --> D([Done])
89
- V -- No --> X[Diagnose and iterate]
90
- X --> F{Same failure x3?}
91
- F -- No --> E
92
- F -- Yes --> Z[Zoom out / re-plan]
93
- Z --> E
75
+ S -- Yes --> K[Kernel: classify + select topology]
76
+ K --> W{Strategy selected?}
77
+ W -- Deferred --> D[Keep invariants<br/>choose smallest justified workflow]
78
+ W -- Selected --> A[ACTIVE workflow contract]
79
+ A --> E[Execute with model-chosen tactics]
80
+ E --> V{Workflow obligations resolved?}
81
+ V -- Yes --> Done([Evidence-backed completion])
82
+ V -- No --> X[Diagnose / re-plan / satisfy missing stage]
83
+ X --> E
94
84
  ```
95
85
 
96
86
  ### Tier 1 — Trivial
97
87
 
98
- Typical triggers: typos, narrow single-file edits, straightforward cleanup, code explanation, simple Git operations.
99
-
100
- Guidance:
101
- - prefer direct execution,
102
- - avoid large plans/delegation,
103
- - when there are no suggestions, use the bounded direct path,
104
- - if a focused skill is suggested, read/evaluate its entry before omitting it,
105
- - still gather enough evidence to support the completion claim.
88
+ Typical triggers: typo/docs correction, narrow local edit, bounded explanation, straightforward Git operation. Usually selects `direct-single`; keep overhead minimal while still satisfying applicable verification/safety rails.
106
89
 
107
90
  ### Tier 2 — Standard
108
91
 
109
- Typical triggers: normal feature work, bug fixes, multi-file changes, security review, focused benchmark/evaluation.
92
+ Typical triggers: normal feature work, bug fixes, multi-file changes, focused benchmark/review. Usually selects `iterative-single`: bounded reason/act work plus objective verification. Focused skills such as `tdd`, `verification-loop`, `security-review`, `using-git-worktrees`, and `eval-harness` may be surfaced and must be resolved for applicability.
110
93
 
111
- Possible suggestions:
112
- - `todo-driven-workflow` when explicit progress tracking helps,
113
- - `tdd` when executable behavioral tests are a useful design/verification tool,
114
- - `verification-loop` when systematic build/lint/test/diff selection helps,
115
- - `security-review` for trust/input/auth/secrets/network boundaries,
116
- - `using-git-worktrees` where isolation reduces workspace risk,
117
- - `eval-harness` for quantitative scoring.
118
-
119
- Every emitted suggestion must be evaluated from its `SKILL.md` flow before omission. After evaluation, none of those imply a universal `TODO → TDD → verification-loop` sequence.
94
+ No universal `TODO → TDD → verification-loop` order is implied. But when a skill is applicable, reading it does not grant permission to ignore its flow.
120
95
 
121
96
  ### Tier 3 — Macro
122
97
 
123
- Typical triggers: repository-wide refactors, architecture/migration work, large ambiguous requirements, broad documentation/system design, or work that may benefit from delegation.
124
-
125
- Possible suggestions:
126
- - `fable-mode` / `fable-discipline` for deliberate macro planning,
127
- - `multi-agent-workspace` for bounded delegation,
128
- - `repo-docs` for project-level documentation,
129
- - `grill-with-docs`, `to-spec`, and `to-tickets` when design decisions/specs/issues genuinely help.
130
-
131
- Tier 3 does **not** automatically mean multi-agent. It does mean that any emitted Tier 3 suggestions are read/evaluated before omission. If one capable agent can complete the work safely and efficiently after that evaluation, direct execution remains valid.
132
-
133
- ## Classifier tuning
98
+ Typical triggers: repository-wide refactors, architecture/migration work, broad synthesis, or bounded delegation. The router may select `fable-staged`, `fable-parallel`, or `fable-multi-agent-workspace`.
134
99
 
135
- `tier-router.js` reads keyword/guide tables from `harness-everything/scripts/routing-keywords.json`. Edit that data to tune Tier 2/3 signals or knowledge-guide matches without changing classifier code.
100
+ A selected Fable topology is mandatory for that run. `fable-parallel` dispatches only validated independent batches; `fable-staged` preserves ordered stage contracts; workspace mode preserves durable handoffs/roles while Fable owns staged execution.
136
101
 
137
- The classifier remains heuristic:
138
- - its recommendation is the default, not an order,
139
- - a clear task reading may override it,
140
- - explicit user intent wins,
141
- - when overriding a materially different tier, record the reason briefly.
102
+ Tier 3 does **not** automatically mean multi-agent. The smallest sufficient topology still wins.
142
103
 
143
104
  ## Pre-action `actionGate`
144
105
 
145
- The router may predict that a request needs `workflowPlan.actionGate`, but the Phase 5 enforcement hook deliberately **re-classifies the exact tool payload**. A missing or incorrect router prediction therefore cannot authorize a destructive command.
106
+ `workflowPlan.actionGate` is orthogonal to topology. The executable hook re-classifies the exact tool payload, so an incorrect router hint cannot authorize a destructive/external side effect.
146
107
 
147
- The executable policy lives in `hooks/scripts/action-gate-rules.json`. A missing or invalid table activates built-in safety defaults and reports the degradation. A valid empty table remains empty so the CI negative control can detect accidental policy erasure.
108
+ Policy lives in `hooks/scripts/action-gate-rules.json`. Matched actions are handled according to host capability and configured policy; approved/executed payload identity is audited. `HARNESS_ACTION_GATE_POLICY=always-ask` forces Claude's ask path where supported. Unsupported host behavior must remain visible rather than being described as equivalent hard enforcement.
148
109
 
149
- | Exact tool action | Harness result | Audit disposition before execution |
150
- |---|---|---|
151
- | Safe unmatched command such as `git status` | allow/no decision | no gate record |
152
- | Matched command on Claude Code (default policy) | no decision: the session's permission mode, rules and auto-mode classifier decide | `deferred-to-host` |
153
- | Matched command on Claude Code with `HARNESS_ACTION_GATE_POLICY=always-ask` | `permissionDecision: "ask"` | `pending-approval` |
154
- | Matched command on a host without Harness-verified ask semantics | block with exit 2 | `rejected` |
155
- | Payload reaches `PostToolUse` | continue | `executed` with exact payload SHA-256 |
156
- | Payload reaches `PostToolUseFailure` | continue | `executed` + failed execution outcome |
157
- | Deferred or pending payload never executes before `Stop` | no execution | `rejected` |
158
- | Hook internal error | Claude: no decision plus a `systemMessage` warning (`always-ask`: `ask`); other/unverified host: exit 2 | host decides on Claude; fail closed elsewhere |
110
+ ## Workflow lifecycle gates
159
111
 
160
- `rm -rf` and `Remove-Item -Recurse -Force` are exempt only when every parsed target is inside the OS temp directory, the current session scratch directory, or an explicitly supplied Harness scratch directory. An uncertain target is gated conservatively.
112
+ The [workflow runtime contract](workflow-runtime.md) defines the executable boundary:
161
113
 
162
- ```mermaid
163
- flowchart TD
164
- P[PreToolUse exact payload] --> C{Matches action-gate rule?}
165
- C -- No --> A[Allow without gate record]
166
- C -- Yes --> X{Scratch-only destructive delete?}
167
- X -- Yes --> A
168
- X -- No --> H{Host has a native permission flow?}
169
- H -- Claude Code --> Y{always-ask policy?}
170
- Y -- No --> D[No decision: the permission mode decides<br/>record deferred-to-host + payload hash]
171
- Y -- Yes --> Q[Emit permissionDecision: ask<br/>record pending-approval + payload hash]
172
- H -- No / unknown --> B[Exit 2 block<br/>record rejected]
173
- D --> U{Did the exact tool payload execute?}
174
- Q --> U
175
- U -- PostToolUse --> E[record executed]
176
- U -- PostToolUseFailure --> F[record executed + failure]
177
- U -- no execution by Stop --> R[record rejected]
178
- ```
114
+ - `workflow-gate.js` checks shell/direct mutation, correlated Fable entry, and major-workflow Git isolation.
115
+ - `workflow-stop-gate.js` checks manifest stage membership and observed verification evidence before `satisfied`.
116
+ - `workflow-disposition.js` starts/replans the run, records a scoped stage escape, or reports `blocked`.
179
117
 
180
- On Claude Code the gate returns no decision by default. The [hooks reference](https://code.claude.com/docs/en/hooks) says a hook's `ask` "also forces a permission prompt in auto mode: the classifier can still deny the tool call, but it can't approve the call silently". A forced `ask` would therefore override the permission mode the user picked (#107).
118
+ A follow-up prompt cannot erase unresolved obligations. Start from the router's displayed session stage-specification path; source mutation waits for entry and isolation. Replan is bounded. Escape requires `workflow-uncovered-scope` or `host-capability-unavailable`, a stage id, uncovered scope, and evidence. It never waives isolation or independent verification.
181
119
 
182
- With no decision, the normal permission flow applies:
120
+ Direct/iterative verification milestones are checked, while semantic check quality and iteration budgeting remain executor obligations. Shell inspection does not contain arbitrary script side effects. Package/mechanism tests do not prove host loading or behavioral compliance; #82 owns retained live evidence.
183
121
 
184
- - Manual and `acceptEdits` prompt unless a rule or the mode already allows the command.
185
- - Auto mode lets the classifier decide.
186
- - `dontAsk` denies anything that is not pre-approved.
187
- - Deny rules apply in every mode.
122
+ ## Cognitive OS relationship
188
123
 
189
- Set `HARNESS_ACTION_GATE_POLICY=always-ask` to have Harness force its own prompt instead. Either way the gate classifies and audits every matched payload.
124
+ `install-cognitive-os` remains the explanatory/manual entry point for Discover → Think → Try → Summarize → Record. It is a reasoning policy, not a peer skill that must be selected before domain work. The selected workflow contract governs lifecycle obligations; the model retains freedom inside it.
190
125
 
191
- The deterministic suite verifies two things: the deferred result for every known `permission_mode` value, and the `ask` JSON under `always-ask`. That is **mechanism evidence only** until preserved live evidence exists.
126
+ ## Enforcement strength
192
127
 
193
- ## Cognitive OS relationship
128
+ A contract can be semantically mandatory even when a host cannot mechanically enforce every transition. Keep those claims separate:
194
129
 
195
- `install-cognitive-os` remains the canonical explanatory/manual entry point for Discover → Think → Try → Summarize → Record. It is **not** required to win peer-skill selection before domain work begins. The runtime kernel establishes the smaller invariant contract independently; domain skills provide their own tactics inside that contract.
130
+ - **contract:** what the agent is required to do;
131
+ - **mechanism:** which transitions a packaged hook/plugin can block or observe;
132
+ - **live evidence:** whether a real host actually loaded/fired that mechanism.
196
133
 
197
- ## Enforcement strength
134
+ Instruction-only integrations receive the same workflow contract as guidance, but must not be labeled hard-enforced. Kernel emission alone does not prove model compliance. #82 owns host-specific retained evidence.
198
135
 
199
- A lifecycle hook can mechanically inject the read-before-skip contract into context, but that alone does not prove the model actually read every suggested `SKILL.md`. Instruction-only integrations can only instruct the model. Do not label either case as live behavioral enforcement without retained host/session evidence; #82 owns that evidence boundary. **Do not infer cross-host parity** from one host's mechanism or live evidence; each host needs its own retained evidence.
136
+ Do not infer cross-host parity from shared runtime code or passing package tests.