harness-everything 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/AGENTS.md +3 -1
- package/CHANGELOG.md +6 -0
- package/README.md +10 -9
- package/ci/invariant-routing.test.js +68 -10
- package/ci/openai-plugin-package.test.js +6 -2
- package/docs/architecture.md +26 -17
- package/docs/mechanism-first-skill-mesh.md +29 -24
- package/docs/philosophy.md +8 -5
- package/docs/platform-capabilities.md +10 -2
- package/docs/routing.md +43 -15
- package/harness-everything/SKILL.md +14 -21
- package/harness-everything/references/triage-and-tiers.md +42 -28
- package/harness-everything/scripts/kernel-router.js +18 -5
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/scripts/lib/advisory-text.js +21 -19
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +14 -21
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +42 -28
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +18 -5
- package/scripts/lib/advisory-text.js +21 -19
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"name": "harness-everything",
|
|
11
11
|
"source": "./",
|
|
12
12
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
13
|
-
"version": "0.
|
|
13
|
+
"version": "0.8.0",
|
|
14
14
|
"author": {
|
|
15
15
|
"name": "Miya Daniel"
|
|
16
16
|
},
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "harness-everything",
|
|
3
3
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.8.0",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Miya Daniel"
|
|
7
7
|
},
|
package/AGENTS.md
CHANGED
|
@@ -4,7 +4,7 @@ description: Operating guide for AI agents working inside the Harness-everything
|
|
|
4
4
|
|
|
5
5
|
# AGENTS.md — Working in This Repository
|
|
6
6
|
|
|
7
|
-
You are modifying **Harness**: an orchestrated agent operating system (skills + hooks/plugins + routing) for AI coding agents. Changes here ship to real agent sessions; a vague `SKILL.md`, stale manifest,
|
|
7
|
+
You are modifying **Harness**: an orchestrated agent operating system (skills + hooks/plugins + routing) for AI coding agents. Changes here ship to real agent sessions; a vague `SKILL.md`, stale manifest, contradictory platform claim, or router suggestion that can be skipped without reading its flow can misroute other people's agents. Precision is the product.
|
|
8
8
|
|
|
9
9
|
## Repository Layout
|
|
10
10
|
|
|
@@ -40,6 +40,7 @@ The retained [OpenCode evidence](benchmarks/results/live-host/opencode-2026-09-1
|
|
|
40
40
|
7. **Runtime/workflow claims are generated facts.** `package.json`, `.nvmrc`, `Dockerfile`, and active `.github/workflows/*.yml` are parsed by `scripts/repository-contract.js`. After changing them, run `npm run docs:sync`; `test:repo-contract` rejects stale generated docs, unsupported action majors, Node runtimes below the support floor, or loss of the Node 22 compatibility lane.
|
|
41
41
|
8. **CHANGELOG.md keeps pending human-authored notes under `[Unreleased]`.** Do not create new alpha/beta/rc release headings; semantic-release owns stable release headings, tags, and release notes.
|
|
42
42
|
9. **Conventional Commit type is release input.** The final commit/squash title that lands on `main` must follow the convention. Breaking changes release major; `feat` releases minor; `fix`, `perf`, `refactor`, `build`, and `revert` release patch; `docs`, `test`, `ci`, `style`, and `chore` do not release by themselves.
|
|
43
|
+
10. **Router suggestions are mandatory to evaluate, advisory to execute.** Before skipping any suggested skill, read its complete `SKILL.md` entry and evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules. A name/description/router summary or “routine task” judgement is not enough. Using one suggestion does not waive read-before-skip for other omissions; unreadable suggestions are `unresolved/unavailable`, not silent skips. Do not turn this into a fixed TODO/TDD/Fable execution pipeline.
|
|
43
44
|
|
|
44
45
|
## Verification Before You Claim Done
|
|
45
46
|
|
|
@@ -56,6 +57,7 @@ npm run test:references # executable/deep-dive references
|
|
|
56
57
|
npm run test:release # release catalog + semantic-release/version-sync contract
|
|
57
58
|
npm run test:collision # description collision detection
|
|
58
59
|
npm run test:routing:skills # every positive skill route reaches its target
|
|
60
|
+
npm run test:routing:invariants # kernel/read-before-skip routing contract
|
|
59
61
|
npm run test:plugin:openai # local OpenAI/Codex plugin package
|
|
60
62
|
npm run test:plugin:submission # public Skills-only submission inputs/bundle parity
|
|
61
63
|
npm run test:platform:compatibility
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project are documented in this file.
|
|
4
4
|
|
|
5
|
+
## [0.8.0](https://github.com/dyphn1/Harness-everything/compare/v0.7.0...v0.8.0) (2026-09-17)
|
|
6
|
+
|
|
7
|
+
### Features
|
|
8
|
+
|
|
9
|
+
* **router:** require evaluation before skipping suggested skills ([#140](https://github.com/dyphn1/Harness-everything/issues/140)) ([b959cb9](https://github.com/dyphn1/Harness-everything/commit/b959cb94afff984c4e4507f0be7f6c62dfa89961)), closes [#131](https://github.com/dyphn1/Harness-everything/issues/131)
|
|
10
|
+
|
|
5
11
|
## [0.7.0](https://github.com/dyphn1/Harness-everything/compare/v0.6.1...v0.7.0) (2026-09-17)
|
|
6
12
|
|
|
7
13
|
### Features
|
package/README.md
CHANGED
|
@@ -19,7 +19,7 @@ AI coding agents are highly capable, but they struggle with self-regulation, env
|
|
|
19
19
|
|
|
20
20
|
Harness acts as an automated system supervisor. It remains completely silent and out of the way, intervening only when execution boundaries are violated or failures are detected.
|
|
21
21
|
|
|
22
|
-
Harness deliberately follows a **minimal rails, maximum freedom** design: the runtime enforces
|
|
22
|
+
Harness deliberately follows a **minimal rails, maximum freedom** design: the runtime enforces route-before-execution, verify-before-claim, and re-plan-after repeated identical failure. When routing emits skill suggestions, it adds one conditional rail: **evaluate before skip**. The agent must read each suggested skill's `SKILL.md` entry/basic flow before omitting it, but remains free to choose, combine, reorder, or skip execution after that evaluation.
|
|
23
23
|
|
|
24
24
|
### Comparison: Prompt vs. Skill vs. Harness
|
|
25
25
|
|
|
@@ -68,7 +68,7 @@ npx github:dyphn1/Harness-everything install
|
|
|
68
68
|
1. **Use the selected surface's real mechanism:** Claude Code hooks, the local OpenAI plugin lifecycle hooks, OpenCode's plugin API, or advisory instructions depending on what you installed.
|
|
69
69
|
2. **Preflight / session context where packaged:** Hook-capable surfaces can inject environment/session context automatically; advisory-only surfaces must not be described as if they do.
|
|
70
70
|
3. **Verification boundary:** Completion claims require objective evidence; whether that boundary is mechanically invoked or explicitly called depends on the host surface.
|
|
71
|
-
4. **
|
|
71
|
+
4. **Mandatory evaluation, advisory execution:** Tier classification may suggest useful skills. Before skipping any suggested skill, the agent reads its complete `SKILL.md` entry/basic flow and evaluates applicability; execution remains agent-controlled afterward, so Harness still does not impose a universal TODO/TDD/Fable sequence.
|
|
72
72
|
|
|
73
73
|
### What Gets Installed (and How to Remove It)
|
|
74
74
|
|
|
@@ -119,9 +119,10 @@ flowchart TD
|
|
|
119
119
|
flowchart TD
|
|
120
120
|
U([User Request]) --> K[Harness Kernel<br/>classify scope + establish invariants]
|
|
121
121
|
K --> T{Tier recommendation}
|
|
122
|
-
T
|
|
123
|
-
|
|
124
|
-
|
|
122
|
+
T --> S{Suggested skills?}
|
|
123
|
+
S -->|Yes| R[Read each suggested SKILL.md<br/>evaluate flow + applicability]
|
|
124
|
+
S -->|No| A[Agent chooses smallest useful tactic / skill set]
|
|
125
|
+
R --> A
|
|
125
126
|
A --> Exec[Execute Code / Run Commands]
|
|
126
127
|
Exec --> Gate{Objective evidence supports completion?}
|
|
127
128
|
Gate -->|No| Retry[Diagnose / iterate]
|
|
@@ -137,7 +138,7 @@ flowchart TD
|
|
|
137
138
|
style Gate fill:#ffcdd2,stroke:#c62828,stroke-width:1px,color:#000000
|
|
138
139
|
```
|
|
139
140
|
|
|
140
|
-
The Tier changes the **recommendations**, not the required order. Tier 2 may suggest `tdd`, `todo-driven-workflow`, or `verification-loop`; Tier 3 may suggest Fable or multi-agent capabilities.
|
|
141
|
+
The Tier changes the **recommendations**, not the required order. Tier 2 may suggest `tdd`, `todo-driven-workflow`, or `verification-loop`; Tier 3 may suggest Fable or multi-agent capabilities. Suggestions are **mandatory to evaluate and advisory to execute**: read each suggested skill's `SKILL.md` flow before omission, then let the model decide what actually helps.
|
|
141
142
|
|
|
142
143
|
---
|
|
143
144
|
|
|
@@ -145,7 +146,7 @@ The Tier changes the **recommendations**, not the required order. Tier 2 may sug
|
|
|
145
146
|
|
|
146
147
|
Harness operates through six core cognitive concepts:
|
|
147
148
|
|
|
148
|
-
1. **Kernel Router (`kernel-router.js` + `tier-router.js`):** `tier-router.js` remains the classifier, dynamic-skill detector, and knowledge-guide matcher. `kernel-router.js` is the public runtime boundary: it preserves the classifier result, removes legacy fixed-pipeline instructions, injects the
|
|
149
|
+
1. **Kernel Router (`kernel-router.js` + `tier-router.js`):** `tier-router.js` remains the classifier, dynamic-skill detector, and knowledge-guide matcher. `kernel-router.js` is the public runtime boundary: it preserves the classifier result, removes legacy fixed-pipeline instructions, injects the baseline invariants, and adds `evaluate-suggestions-before-skip` whenever domain skills are suggested. Suggestions are mandatory to evaluate by reading their `SKILL.md` entry/basic flow, but advisory to execute after evaluation. This prevents host peer-skill selection or superficial routing summaries from bypassing Harness while preserving agent autonomy. If nothing matches at all — including nothing already kept from the open skills ecosystem — `find-skills` checks `npx skills list` live and, if still nothing, searches `skills.sh`/`npx skills` with explicit approval before installation.
|
|
149
150
|
2. **Guard (`rule-of-3.js`):** The fail-safe circuit breaker. Tracks failure signatures across terminal runs on integration surfaces that package the required lifecycle hooks. If a test or command fails 3 times with the same signature, it locks mutating tools and forces a `zoom-out` reflection: re-verify every assumption with read-only tools, write a fact-checked report, then resume on a fresh diagnosis. A companion `Stop` gate (`stop-gate.js`) can bounce the end of a turn once per edit batch when edits were never followed by successful verification on hosts where that hook is installed.
|
|
150
151
|
3. **Memory (`state-persist.js`):** Session transaction logging for stateful hook/plugin integrations. Static skills/instructions alone do not create WAL state.
|
|
151
152
|
4. **Reflection (`self-evolve`):** Long-term workspace immunization. Upon task completion, the agent reflects on the root cause of resolved issues, then judges whether the lesson is a simple rule or a reusable, complex pattern: simple rules are appended to local workspace rules (`RULES.md`); genuinely reusable patterns are instead packaged as a dynamic skill (via `skill-creator`'s Dynamic Skill Generation Contract) and registered in `manifest.json` so the Router picks it up in future sessions. Either path is validated by a hermetic self-regression suite before it's persisted.
|
|
@@ -285,7 +286,7 @@ Mechanism tests prove individual packaged mechanisms; only real host/session evi
|
|
|
285
286
|
|
|
286
287
|
### Catalog hygiene
|
|
287
288
|
|
|
288
|
-
`npm run test:consistency` keeps distribution manifests, docs links, skill frontmatter, routing-eval coverage, platform capability claims, and the generated repository runtime/workflow contract in lockstep with what is actually on disk. `npm run test:repo-contract` runs the runtime/workflow drift gate directly, while `npm run docs:sync` regenerates [docs/repository-contract.md](docs/repository-contract.md) after an intentional change. `npm run test:docs:capabilities` runs the platform-doc drift check directly. `npm run test:references` checks every executable/deep-dive path named by `SKILL.md`; `npm run test:release` compares release/catalog evidence; `npm run test:routing:skills` recursively classifies nested skills and executes the real router for every directly-routable skill's positive cases; `npm run test:routing:invariants` guards the invariant-first architecture; and `harness verify-install` detects stale installed versions or file trees. The installer E2E gate performs install → verify-install → uninstall against seeded user-owned files on Ubuntu, Windows, and macOS so path and ownership symmetry regressions fail CI. `npm run test:collision` fails CI when two skills' descriptions overlap enough to confuse the router.
|
|
289
|
+
`npm run test:consistency` keeps distribution manifests, docs links, skill frontmatter, routing-eval coverage, platform capability claims, and the generated repository runtime/workflow contract in lockstep with what is actually on disk. `npm run test:repo-contract` runs the runtime/workflow drift gate directly, while `npm run docs:sync` regenerates [docs/repository-contract.md](docs/repository-contract.md) after an intentional change. `npm run test:docs:capabilities` runs the platform-doc drift check directly. `npm run test:references` checks every executable/deep-dive path named by `SKILL.md`; `npm run test:release` compares release/catalog evidence; `npm run test:routing:skills` recursively classifies nested skills and executes the real router for every directly-routable skill's positive cases; `npm run test:routing:invariants` guards the invariant-first architecture including read-before-skip; and `harness verify-install` detects stale installed versions or file trees. The installer E2E gate performs install → verify-install → uninstall against seeded user-owned files on Ubuntu, Windows, and macOS so path and ownership symmetry regressions fail CI. `npm run test:collision` fails CI when two skills' descriptions overlap enough to confuse the router.
|
|
289
290
|
|
|
290
291
|
---
|
|
291
292
|
|
|
@@ -311,4 +312,4 @@ npm run test:plugin:openai
|
|
|
311
312
|
npm run test:plugin:submission
|
|
312
313
|
```
|
|
313
314
|
|
|
314
|
-
After intentional runtime/workflow changes, run `npm run docs:sync` and commit the regenerated repository contract. All script modifications must pass 100% cleanly before pushing to keep the runtime immunized against behavioral regression.
|
|
315
|
+
After intentional runtime/workflow changes, run `npm run docs:sync` and commit the regenerated repository contract. All script modifications must pass 100% cleanly before pushing to keep the runtime immunized against behavioral regression.
|
|
@@ -35,7 +35,9 @@ check(tier2.stdout.includes('Route before execution'), 'route-before-execution i
|
|
|
35
35
|
check(tier2.stdout.includes('Verify before claim'), 'verify-before-claim invariant is present');
|
|
36
36
|
check(tier2.stdout.includes('after 3 same-signature failures'), 'failure escalation invariant is present');
|
|
37
37
|
check(tier2.stdout.includes('loop-budget:'), 'iterative-single loop budget invariant is present');
|
|
38
|
-
check(tier2.stdout.includes('
|
|
38
|
+
check(tier2.stdout.includes('evaluate-suggestions-before-skip'), 'non-empty suggestions inject evaluate-before-skip invariant');
|
|
39
|
+
check(tier2.stdout.includes('SUGGESTED SKILLS (MANDATORY EVALUATION'), 'domain suggestions are mandatory to evaluate');
|
|
40
|
+
check(tier2.stdout.includes('ADVISORY EXECUTION'), 'domain suggestion execution remains advisory');
|
|
39
41
|
check(tier2.stdout.includes('tdd:'), 'Tier 2 can still recommend TDD');
|
|
40
42
|
check(!tier2.stdout.includes('BASE EXECUTION LOOP'), 'legacy fixed base-execution-loop output is suppressed');
|
|
41
43
|
check(tier2.stdout.includes('Do not enforce workflow order'), 'self-orchestration policy is explicit');
|
|
@@ -46,7 +48,13 @@ check(tier2.stdout.includes('- Tier: Tier 2'), 'checkpoint exposes the human-rea
|
|
|
46
48
|
check(tier2.stdout.includes('- Strategy: iterative-single'), 'checkpoint exposes the selected strategy');
|
|
47
49
|
check(/- Required invariants: .*verify-before-claim/.test(tier2.stdout), 'checkpoint exposes required invariants');
|
|
48
50
|
check(/- Suggested skills: .*tdd/.test(tier2.stdout), 'checkpoint exposes deduplicated suggestions');
|
|
49
|
-
check(tier2.stdout.includes('
|
|
51
|
+
check(tier2.stdout.includes('Suggestion evaluation: MANDATORY'), 'checkpoint makes suggestion evaluation mandatory');
|
|
52
|
+
check(tier2.stdout.includes('read its complete SKILL.md entry'), 'checkpoint requires reading the suggested skill entry before skip');
|
|
53
|
+
check(tier2.stdout.includes('name, description, router summary'), 'checkpoint rejects metadata-only skip decisions');
|
|
54
|
+
check(tier2.stdout.includes('routine/common task'), 'checkpoint rejects generic routine-task skip decisions');
|
|
55
|
+
check(tier2.stdout.includes('Using one suggestion does not waive read-before-skip'), 'checkpoint applies read-before-skip per omitted suggestion');
|
|
56
|
+
check(tier2.stdout.includes('unresolved/unavailable rather than skipped'), 'checkpoint preserves unresolved suggestions instead of silent skip');
|
|
57
|
+
check(tier2.stdout.includes('If all are skipped after evaluation'), 'all-skipped rationale is grounded after evaluation');
|
|
50
58
|
|
|
51
59
|
const tier3 = spawnSync(process.execPath, [kernel, 'audit the entire repository architecture and coordinate multiple modules'], {
|
|
52
60
|
cwd: ROOT,
|
|
@@ -56,15 +64,29 @@ check(tier3.status === 0, 'Tier 3 kernel routing exits successfully');
|
|
|
56
64
|
check(/RECOMMENDED TIER:\s*Tier 3/i.test(tier3.stdout), 'macro task remains Tier 3');
|
|
57
65
|
check(tier3.stdout.includes('"strategy":"fable-staged"'), 'Tier 3 dependent/unproven work selects fable-staged');
|
|
58
66
|
check(tier3.stdout.includes('fable-mode / fable-discipline'), 'Tier 3 selected plan suggests Fable capabilities');
|
|
59
|
-
check(tier3.stdout.includes('
|
|
67
|
+
check(tier3.stdout.includes('execution remains advisory after evaluation'), 'Tier 3 execution remains non-mandatory after evaluation');
|
|
68
|
+
check(tier3.stdout.includes('evaluate-suggestions-before-skip'), 'Tier 3 suggestions also carry read-before-skip invariant');
|
|
60
69
|
check(tier3.stdout.includes('HARNESS ROUTING CHECKPOINT (REQUIRED VISIBLE STATE)'), 'Tier 3 also emits the required routing checkpoint');
|
|
61
70
|
|
|
62
71
|
const harnessSkill = read('harness-everything/SKILL.md');
|
|
63
72
|
check(!/Requests that already name a skill/i.test(harnessSkill), 'harness entrypoint no longer opts out when another skill is named');
|
|
64
|
-
check(/
|
|
73
|
+
check(/Work that names or matches another skill/i.test(harnessSkill), 'named domain skills still pass through Harness routing');
|
|
65
74
|
check(/not a fixed pipeline/i.test(harnessSkill), 'skill contract rejects fixed pipeline semantics');
|
|
66
|
-
check(/
|
|
67
|
-
check(/
|
|
75
|
+
check(/Harness Routing Checkpoint\*\*.*user-visible/i.test(harnessSkill), 'skill contract requires checkpoint visibility');
|
|
76
|
+
check(/For every suggestion, read its `SKILL\.md`/i.test(harnessSkill), 'skill contract requires reading each suggested skill before skip');
|
|
77
|
+
check(/`USE FOR`, `DO NOT USE FOR`, workflow\/basic flow, and hard rules/i.test(harnessSkill), 'skill contract defines the required evaluation scope');
|
|
78
|
+
check(/Name, description, router summary, or "routine\/common task" alone cannot justify skip/i.test(harnessSkill), 'skill contract rejects metadata-only omission');
|
|
79
|
+
check(/Using one suggestion does not waive evaluation of others/i.test(harnessSkill), 'skill contract applies evaluation per omitted suggestion');
|
|
80
|
+
check(/Unreadable suggestions are `unresolved\/unavailable`/i.test(harnessSkill), 'skill contract defines unreadable suggestion disposition');
|
|
81
|
+
check(/If all are skipped, give one brief flow-grounded reason/i.test(harnessSkill), 'skill contract preserves evaluated all-skipped rationale');
|
|
82
|
+
check(/mandatory evaluation, advisory execution/i.test(harnessSkill), 'skill entry states the core disposition succinctly');
|
|
83
|
+
|
|
84
|
+
const pluginHarnessSkill = read('plugins/harness-everything/skills/harness-everything/SKILL.md');
|
|
85
|
+
check(pluginHarnessSkill === harnessSkill, 'canonical/plugin Harness SKILL.md copies remain identical');
|
|
86
|
+
const pluginKernel = read('plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js');
|
|
87
|
+
check(pluginKernel === read('harness-everything/scripts/kernel-router.js'), 'canonical/plugin kernel-router copies remain identical');
|
|
88
|
+
const pluginTriage = read('plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md');
|
|
89
|
+
check(pluginTriage === read('harness-everything/references/triage-and-tiers.md'), 'canonical/plugin triage references remain identical');
|
|
68
90
|
|
|
69
91
|
const cognitiveSkill = read('install-cognitive-os/SKILL.md');
|
|
70
92
|
check(/policy, not a required peer-skill selection/i.test(cognitiveSkill), 'Cognitive OS is defined as policy rather than peer-skill dependency');
|
|
@@ -76,14 +98,50 @@ const command = promptHooks && promptHooks[0] && promptHooks[0].hooks && promptH
|
|
|
76
98
|
check(command === 'node "${CLAUDE_PLUGIN_ROOT}/harness-everything/scripts/kernel-router.js"', 'UserPromptSubmit anchors the Harness kernel to the Claude plugin root');
|
|
77
99
|
|
|
78
100
|
const triageDoc = read('harness-everything/references/triage-and-tiers.md');
|
|
79
|
-
check(triageDoc.includes('Do not enforce workflow order. Enforce workflow invariants.'), 'architecture reference records
|
|
101
|
+
check(triageDoc.includes('Do not enforce workflow order. Enforce workflow invariants and evaluate suggestions before skipping them.'), 'architecture reference records invariant-first plus read-before-skip decision');
|
|
80
102
|
check(triageDoc.includes('A tier is **not** a fixed pipeline'), 'tier documentation rejects rigid orchestration');
|
|
81
103
|
check(!triageDoc.includes('MUST load `todo-driven-workflow`'), 'documentation no longer mandates TODO as universal Tier 2/3 first step');
|
|
82
104
|
check(triageDoc.includes('```mermaid'), 'updated architecture retains executable flowchart documentation');
|
|
83
105
|
check(triageDoc.includes('the kernel always emits a compact checkpoint'), 'reference makes checkpoint emission mandatory');
|
|
84
|
-
check(
|
|
85
|
-
check(triageDoc.includes('
|
|
86
|
-
check(triageDoc.includes('
|
|
106
|
+
check(triageDoc.includes('mandatory to **evaluate**'), 'reference requires evaluation of every emitted suggestion');
|
|
107
|
+
check(triageDoc.includes('Using one suggested skill does not waive evaluation'), 'reference preserves per-suggestion evaluation');
|
|
108
|
+
check(triageDoc.includes('cannot be resolved/read is not a valid skip'), 'reference preserves unresolved/unavailable state');
|
|
109
|
+
check(triageDoc.includes('Kernel emission alone is not proof'), 'reference preserves the #82 host-evidence boundary');
|
|
110
|
+
|
|
111
|
+
const routingDoc = read('docs/routing.md');
|
|
112
|
+
check(routingDoc.includes('Suggested skills are mandatory to evaluate and advisory to execute.'), 'routing docs state mandatory-evaluation/advisory-execution contract');
|
|
113
|
+
check(routingDoc.includes('complete `SKILL.md` entry'), 'routing docs require complete skill entry reads before omission');
|
|
114
|
+
check(routingDoc.includes('“routine/common task.”'), 'routing docs reject routine-task skip shortcut');
|
|
115
|
+
|
|
116
|
+
const architectureDoc = read('docs/architecture.md');
|
|
117
|
+
check(architectureDoc.includes('Mandatory evaluation, advisory execution'), 'architecture docs expose the new routing contract');
|
|
118
|
+
check(architectureDoc.includes('evaluate-suggestions-before-skip'), 'architecture docs name the conditional invariant');
|
|
119
|
+
|
|
120
|
+
const philosophyDoc = read('docs/philosophy.md');
|
|
121
|
+
check(philosophyDoc.includes('Mandatory Evaluation, Advisory Execution'), 'philosophy docs preserve autonomy after informed evaluation');
|
|
122
|
+
|
|
123
|
+
const meshDoc = read('docs/mechanism-first-skill-mesh.md');
|
|
124
|
+
check(meshDoc.includes('mandatory to evaluate before omission, but advisory to execute after evaluation'), 'skill-mesh docs prevent advisory suggestions from becoming dead code');
|
|
125
|
+
|
|
126
|
+
const capabilitiesDoc = read('docs/platform-capabilities.md');
|
|
127
|
+
check(capabilitiesDoc.includes('repository/agent contract'), 'platform docs distinguish routing contract from host enforcement evidence');
|
|
128
|
+
check(capabilitiesDoc.includes('does not change `platform-compatibility.json` status values'), 'routing change does not inflate compatibility status');
|
|
129
|
+
|
|
130
|
+
const readme = read('README.md');
|
|
131
|
+
check(readme.includes('**evaluate before skip**'), 'README describes the conditional evaluation rail');
|
|
132
|
+
check(readme.includes('mandatory to evaluate and advisory to execute'), 'README explains suggestion evaluation semantics');
|
|
133
|
+
|
|
134
|
+
const agents = read('AGENTS.md');
|
|
135
|
+
check(agents.includes('Router suggestions are mandatory to evaluate, advisory to execute.'), 'AGENTS.md requires read-before-skip for repository agents');
|
|
136
|
+
|
|
137
|
+
const advisory = read('scripts/lib/advisory-text.js');
|
|
138
|
+
check(advisory.includes('For EVERY suggested skill'), 'generated advisory instructions require per-suggestion evaluation');
|
|
139
|
+
check(advisory.includes('entry/basic flow before skipping it'), 'Codex AGENTS generator includes read-before-skip');
|
|
140
|
+
check(advisory.includes('There is NO universal TODO/TDD/Fable sequence'), 'advisory surfaces reject the old fixed execution pipeline');
|
|
141
|
+
check(!advisory.includes('Initialize the `todo-driven-workflow` checklist first'), 'old forced Tier 2/3 checklist pipeline is removed');
|
|
142
|
+
check(!advisory.includes('load `fable-mode` and `fable-discipline`, run sub-agents'), 'old forced Tier 3 Fable pipeline is removed');
|
|
143
|
+
const pluginAdvisory = read('plugins/harness-everything/scripts/lib/advisory-text.js');
|
|
144
|
+
check(pluginAdvisory === advisory, 'OpenAI plugin advisory mirror matches canonical source');
|
|
87
145
|
|
|
88
146
|
console.log(`\n${failed === 0 ? 'PASS' : 'FAIL'}: invariant routing contract (${failed} failure${failed === 1 ? '' : 's'})`);
|
|
89
147
|
process.exit(failed === 0 ? 0 : 1);
|
|
@@ -164,14 +164,18 @@ assert.strictEqual(second.status, 0, second.stderr);
|
|
|
164
164
|
assert.strictEqual(first.stdout, second.stdout, 'identical routing input must produce identical output');
|
|
165
165
|
assert.match(first.stdout, /RECOMMENDED TIER:\s*Tier 2/i);
|
|
166
166
|
assert.match(first.stdout, /REQUIRED HARNESS INVARIANTS/);
|
|
167
|
-
assert.match(first.stdout, /SUGGESTED SKILLS \(ADVISORY/);
|
|
167
|
+
assert.match(first.stdout, /SUGGESTED SKILLS \(MANDATORY EVALUATION .*ADVISORY EXECUTION\)/);
|
|
168
|
+
assert.match(first.stdout, /evaluate-suggestions-before-skip/);
|
|
169
|
+
assert.match(first.stdout, /read its complete SKILL\.md entry/);
|
|
168
170
|
assert.doesNotMatch(first.stdout, /BASE EXECUTION LOOP/);
|
|
169
171
|
assert.match(first.stdout, /Do not enforce workflow order/);
|
|
170
172
|
|
|
171
173
|
const trivial = route('Update one README typo');
|
|
172
174
|
assert.strictEqual(trivial.status, 0, trivial.stderr);
|
|
173
175
|
assert.match(trivial.stdout, /RECOMMENDED TIER:\s*Tier 1/i);
|
|
174
|
-
assert.match(trivial.stdout, /
|
|
176
|
+
assert.match(trivial.stdout, /Suggested skills: none/);
|
|
177
|
+
assert.match(trivial.stdout, /No domain skill was suggested/);
|
|
178
|
+
assert.doesNotMatch(trivial.stdout, /evaluate-suggestions-before-skip/);
|
|
175
179
|
|
|
176
180
|
const unmatched = route('frobnicate the quux');
|
|
177
181
|
assert.strictEqual(unmatched.status, 0, unmatched.stderr);
|
package/docs/architecture.md
CHANGED
|
@@ -11,13 +11,16 @@ Harness is a **system supervisor**, not a universal workflow engine. Skills rema
|
|
|
11
11
|
The architecture follows a mechanism-first skill mesh with a **minimal kernel**:
|
|
12
12
|
|
|
13
13
|
- classify scope before mutation,
|
|
14
|
+
- when the router suggests skills, require the agent to read/evaluate each suggested `SKILL.md` before omission,
|
|
14
15
|
- require evidence before completion claims,
|
|
15
16
|
- stop repeated same-signature micro-retries and re-plan,
|
|
16
|
-
- leave domain
|
|
17
|
+
- leave domain-skill execution, ordering, planning style, and delegation to the agent after evaluation.
|
|
17
18
|
|
|
18
19
|
> **Do not enforce workflow order. Enforce workflow invariants.**
|
|
20
|
+
>
|
|
21
|
+
> **Mandatory evaluation, advisory execution:** a suggested skill must be read/evaluated before skip, but need not be executed when its flow does not apply.
|
|
19
22
|
|
|
20
|
-
This is intentionally different from the older design where Tier 2 implied a fixed TDD/checklist pipeline and Tier 3 implied a mandatory Fable/multi-agent pipeline.
|
|
23
|
+
This is intentionally different from the older design where Tier 2 implied a fixed TDD/checklist pipeline and Tier 3 implied a mandatory Fable/multi-agent pipeline. It is also stronger than a pure recommendation system where the model can ignore suggestions from their names alone.
|
|
21
24
|
|
|
22
25
|
```mermaid
|
|
23
26
|
flowchart TD
|
|
@@ -29,8 +32,11 @@ flowchart TD
|
|
|
29
32
|
Boot --> Preflight[Environment / integration discovery]
|
|
30
33
|
Preflight --> KR[kernel-router.js]
|
|
31
34
|
KR --> TR[tier-router.js<br/>classifier + guide discovery]
|
|
32
|
-
TR --> Inv[Inject
|
|
33
|
-
Inv -->
|
|
35
|
+
TR --> Inv[Inject invariants]
|
|
36
|
+
Inv --> Sug{Suggested skills?}
|
|
37
|
+
Sug -- Yes --> Eval[Read each suggested SKILL.md<br/>evaluate flow + applicability]
|
|
38
|
+
Sug -- No --> Choice[Agent chooses useful tactics]
|
|
39
|
+
Eval --> Choice
|
|
34
40
|
end
|
|
35
41
|
|
|
36
42
|
subgraph Execution [Agent-Controlled Execution]
|
|
@@ -61,10 +67,13 @@ flowchart TD
|
|
|
61
67
|
`kernel-router.js` is the public runtime entry point. It delegates heuristic classification and dynamic guide/skill discovery to `tier-router.js`, then converts the result into the Harness contract:
|
|
62
68
|
|
|
63
69
|
- recommended Tier + rationale,
|
|
64
|
-
- required invariants,
|
|
65
|
-
-
|
|
70
|
+
- baseline required invariants,
|
|
71
|
+
- a conditional `evaluate-suggestions-before-skip` invariant whenever suggestions exist,
|
|
72
|
+
- skill suggestions under a **mandatory evaluation / advisory execution** contract.
|
|
66
73
|
|
|
67
|
-
|
|
74
|
+
For each suggested skill, the agent must read the complete `SKILL.md` entry and evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules before omission. A skill name, description, router summary, tier label, or “routine task” judgement is not enough to justify skipping. If the entry explicitly requires another document to determine applicability, that required material joins the evaluation; optional deep-dive references do not automatically become mandatory.
|
|
75
|
+
|
|
76
|
+
The kernel deliberately suppresses the old fixed `BASE EXECUTION LOOP` wording so that a host selecting `tdd`, `security-review`, `repo-docs`, or another peer skill cannot accidentally replace or bypass the cross-cutting Harness contract. Using one suggested skill also does not waive evaluation of the other omitted suggestions.
|
|
68
77
|
|
|
69
78
|
`harness-everything` remains the public/manual skill entry point for routing, debugging, and re-routing. `install-cognitive-os` remains the explanatory/manual entry point for the Discover → Think → Try → Summarize → Record policy. Automatic correctness must not depend on either one winning host peer-skill selection first.
|
|
70
79
|
|
|
@@ -93,12 +102,12 @@ Runtime state includes hook metadata, circuit-breaker counters, handoff/verifica
|
|
|
93
102
|
The installer configures native lifecycle hooks and project skills.
|
|
94
103
|
|
|
95
104
|
- `SessionStart`: `bootstrap.js` restores prior state and audits integrations.
|
|
96
|
-
- `UserPromptSubmit`: `kernel-router.js` establishes routing + invariants before peer/domain skill execution.
|
|
105
|
+
- `UserPromptSubmit`: `kernel-router.js` establishes routing + invariants before peer/domain skill execution and tells the agent that every suggested skill must be evaluated before omission.
|
|
97
106
|
- `PreToolUse`: circuit breaker, boundary/depth/context guards, and subagent scope guards can block supported tool calls.
|
|
98
107
|
- `PostToolUse`: records outcomes/state and tracks repeated failures.
|
|
99
108
|
- `Stop`: `stop-gate.js` prevents an edit batch from being claimed complete without successful verification evidence.
|
|
100
109
|
|
|
101
|
-
The prompt hook is intentionally lightweight: it does **not** prescribe TODO/TDD/Fable order. It
|
|
110
|
+
The prompt hook is intentionally lightweight: it does **not** prescribe TODO/TDD/Fable order. It injects the routing rails and the read-before-skip requirement, then lets the model orchestrate execution after evaluation. Kernel emission by itself is not proof that the host/model actually read each skill; that requires retained live behavioral evidence.
|
|
102
111
|
|
|
103
112
|
### 2. OpenCode — plugin enforcement, scoped live verification
|
|
104
113
|
|
|
@@ -106,20 +115,20 @@ The prompt hook is intentionally lightweight: it does **not** prescribe TODO/TDD
|
|
|
106
115
|
|
|
107
116
|
### 3. Cursor — skills plus advisory rules
|
|
108
117
|
|
|
109
|
-
The current installer uses `.cursor/skills/` plus `.cursorrules`. Cursor’s documented skill and plugin surfaces are separate from Harness runtime hooks; routing/verification/retry boundaries remain
|
|
118
|
+
The current installer uses `.cursor/skills/` plus `.cursorrules`. Cursor’s documented skill and plugin surfaces are separate from Harness runtime hooks; routing/verification/retry boundaries and read-before-skip remain instruction-governed unless a compatible mechanism is independently verified.
|
|
110
119
|
|
|
111
120
|
### 4. GitHub Copilot agent surfaces — skills plus advisory instructions
|
|
112
121
|
|
|
113
|
-
The current installer uses `.github/skills/` plus `.github/copilot-instructions.md`. GitHub’s Agent Skills paths are documented, but no Harness-specific plugin or live session is verified here.
|
|
122
|
+
The current installer uses `.github/skills/` plus `.github/copilot-instructions.md`. GitHub’s Agent Skills paths are documented, but no Harness-specific plugin or live session is verified here. The generated instructions include read-before-skip, but repository text alone is not live compliance evidence.
|
|
114
123
|
|
|
115
124
|
### 5. Codex — local OpenAI plugin plus advisory installer path
|
|
116
125
|
|
|
117
126
|
Codex has two distinct Harness integration surfaces and documentation must keep them separate:
|
|
118
127
|
|
|
119
|
-
- The general `--codex` installer path writes Codex-facing skills/instructions such as `AGENTS.md`; where the host only consumes those instructions, that path is advisory.
|
|
128
|
+
- The general `--codex` installer path writes Codex-facing skills/instructions such as `AGENTS.md`; where the host only consumes those instructions, that path is advisory. The generated `AGENTS.md` tells the agent to evaluate every router-suggested skill entry before omission.
|
|
120
129
|
- The packaged local OpenAI plugin under `plugins/harness-everything/` includes `.codex-plugin/plugin.json`, all 26 canonical skills, and lifecycle hooks for `SessionStart`, `UserPromptSubmit`, supported local `PreToolUse` / `PostToolUse`, `SubagentStart` / `SubagentStop`, and `Stop`. Those hooks inject the compact session policy, run the invariant-first kernel, track supported edits and verification, and surface subagent scope changes.
|
|
121
130
|
|
|
122
|
-
The local plugin therefore has **mechanism-tested local enforcement** for the packaged event/tool mappings. This is package evidence, not a live-host claim; the host's hook review/trust flow and a fresh session are still required. Do not infer broader Claude parity for events or tools that are not listed in the package.
|
|
131
|
+
The local plugin therefore has **mechanism-tested local enforcement** for the packaged event/tool mappings. This is package evidence, not a live-host claim; the host's hook review/trust flow and a fresh session are still required. Do not infer broader Claude parity for events or tools that are not listed in the package, and do not infer that mechanism-tested prompt injection proves the model complied with read-before-skip.
|
|
123
132
|
|
|
124
133
|
The public OpenAI **Skills-only** submission is narrower again: it ships reusable skills and referenced assets, not the local `.codex-plugin` lifecycle hooks. Public reviewer/listing claims must describe skill/workflow behavior rather than local hook enforcement. See [openai-plugin.md](openai-plugin.md).
|
|
125
134
|
|
|
@@ -148,9 +157,9 @@ flowchart LR
|
|
|
148
157
|
Z --> T
|
|
149
158
|
```
|
|
150
159
|
|
|
151
|
-
A domain skill may have its own lifecycle — for example RED/GREEN/REFACTOR inside `tdd` — without being forced into a global sequence. The
|
|
160
|
+
A domain skill may have its own lifecycle — for example RED/GREEN/REFACTOR inside `tdd` — without being forced into a global sequence. The shared obligations are the baseline kernel invariants plus read-before-skip when router suggestions exist.
|
|
152
161
|
|
|
153
|
-
This separation solves the peer-skill routing problem: a strong host may correctly decide that a task primarily needs `tdd` or `security-review`; that choice is allowed. What it cannot silently erase is the routing/evidence/retry contract
|
|
162
|
+
This separation solves the peer-skill routing problem: a strong host may correctly decide that a task primarily needs `tdd` or `security-review`; that choice is allowed. What it cannot silently erase is the routing/evidence/retry contract or dismiss other suggestions without evaluating their actual skill flows.
|
|
154
163
|
|
|
155
164
|
---
|
|
156
165
|
|
|
@@ -171,9 +180,9 @@ Harness runs with a local-first, zero-trust model:
|
|
|
171
180
|
Static configuration is not enough to claim that a host behaves correctly. Validation is layered:
|
|
172
181
|
|
|
173
182
|
- syntax/reference/manifest checks prove package integrity,
|
|
174
|
-
- `ci/invariant-routing.test.js` proves the kernel
|
|
183
|
+
- `ci/invariant-routing.test.js` proves the kernel, read-before-skip, and documentation contracts,
|
|
175
184
|
- `ci/doc-capability-consistency.test.js` prevents current platform capability claims from drifting apart,
|
|
176
185
|
- mechanism tests prove supported hook/plugin behavior,
|
|
177
|
-
- live host sessions are required before promoting a source/mechanism-tested adapter to live host enforcement.
|
|
186
|
+
- live host sessions are required before promoting a source/mechanism-tested adapter to live host enforcement or claiming that agents actually followed the evaluation contract.
|
|
178
187
|
|
|
179
188
|
The architecture and its Mermaid diagrams are part of that contract. Runtime changes that alter orchestration must update these documents in the same change.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Harness is not intended to be a rigid workflow engine. Its skills should remain independently useful, while still coordinating through lightweight mechanisms that help the model and the software engineer work together with less drift, fewer loops, and better evidence.
|
|
4
4
|
|
|
5
|
-
The preferred shape is a **mechanism-first skill mesh**: skills are autonomous, coordination is explicit, and enforcement happens at narrow decision points through scripts, hooks, exit codes, and compact return values. Platform-specific enforcement claims follow [platform-capabilities.md](platform-capabilities.md).
|
|
5
|
+
The preferred shape is a **mechanism-first skill mesh**: skills are autonomous to execute, coordination is explicit, and enforcement happens at narrow decision points through scripts, hooks, exit codes, and compact return values. Router-suggested skills are **mandatory to evaluate before omission, but advisory to execute after evaluation**. Platform-specific enforcement claims follow [platform-capabilities.md](platform-capabilities.md).
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -17,7 +17,7 @@ Each skill should be able to answer four questions on its own:
|
|
|
17
17
|
3. **What mechanism do I expose?** When possible, the skill should provide a script, command, checklist primitive, verifier, or return-value contract that makes its guidance concrete.
|
|
18
18
|
4. **What should I hand to nearby skills?** The skill may recommend a next skill, but it should not require a global handoff template to remain useful.
|
|
19
19
|
|
|
20
|
-
This keeps the system composable: `tdd` can run alone, `verification-loop` can run alone, `todo-driven-workflow` can run alone, and `harness-everything` can route among them when the task benefits from orchestration.
|
|
20
|
+
This keeps the system composable: `tdd` can run alone, `verification-loop` can run alone, `todo-driven-workflow` can run alone, and `harness-everything` can route among them when the task benefits from orchestration. The router may not execute every suggestion, but it may not discard suggestions without first reading/evaluating their actual `SKILL.md` flows.
|
|
21
21
|
|
|
22
22
|
---
|
|
23
23
|
|
|
@@ -30,7 +30,7 @@ A universal handoff block makes workflow state easy to parse, but it also has co
|
|
|
30
30
|
- It can make the model optimize for satisfying the template rather than making the best next engineering move.
|
|
31
31
|
- It adds prompt weight on platforms where the mechanism cannot actually enforce the block.
|
|
32
32
|
|
|
33
|
-
Harness should avoid mandatory global handoff text. Local skills may still use concise reports when they need them, but the mesh should not depend on one universal response format.
|
|
33
|
+
Harness should avoid mandatory global handoff text. Local skills may still use concise reports when they need them, but the mesh should not depend on one universal response format. The routing checkpoint is a compact state/evaluation surface, not a global handoff protocol.
|
|
34
34
|
|
|
35
35
|
---
|
|
36
36
|
|
|
@@ -42,7 +42,7 @@ Good mechanism outputs are:
|
|
|
42
42
|
|
|
43
43
|
- **Short:** the result should be easy for the model to keep in active attention.
|
|
44
44
|
- **Actionable:** the output should name the next constraint, gate, or recommended skill.
|
|
45
|
-
- **Non-authoritarian:** the mechanism
|
|
45
|
+
- **Non-authoritarian about execution:** the mechanism may require an evaluation step without forcing a universal workflow sequence.
|
|
46
46
|
- **Portable:** hook-capable platforms can run the mechanism automatically; hook-less platforms can call the same script explicitly.
|
|
47
47
|
|
|
48
48
|
Example shape:
|
|
@@ -50,12 +50,13 @@ Example shape:
|
|
|
50
50
|
```text
|
|
51
51
|
HARNESS_DECISION
|
|
52
52
|
tier: 2
|
|
53
|
-
required: route-before-execution, verify-before-claim, re-plan-after-repeat-failure
|
|
53
|
+
required: route-before-execution, verify-before-claim, re-plan-after-repeat-failure, evaluate-suggestions-before-skip
|
|
54
54
|
suggest: todo-driven-workflow, verification-loop
|
|
55
|
+
suggestion_policy: mandatory-evaluation/advisory-execution
|
|
55
56
|
reason: user requested code changes
|
|
56
57
|
```
|
|
57
58
|
|
|
58
|
-
This is not a handoff block. It is a compact mechanical signal. The model remains free to decide how to present the work, explain trade-offs, and collaborate with the engineer.
|
|
59
|
+
This is not a handoff block. It is a compact mechanical signal. The model remains free to decide how to present the work, explain trade-offs, and collaborate with the engineer after it has evaluated the suggested skill flows.
|
|
59
60
|
|
|
60
61
|
---
|
|
61
62
|
|
|
@@ -71,7 +72,7 @@ Recommended contract:
|
|
|
71
72
|
- **Neighbor links:** when another skill is commonly needed, recommend it by name and explain why.
|
|
72
73
|
- **Exit discipline:** if the skill reaches a blocking state, it should say what evidence is missing or what human decision is needed.
|
|
73
74
|
|
|
74
|
-
|
|
75
|
+
The local workflow is why read-before-skip can work: the agent can inspect a suggested skill's complete entry and basic flow to judge applicability without loading the whole ecosystem. Optional deep references remain optional unless that entry explicitly makes one necessary to determine applicability.
|
|
75
76
|
|
|
76
77
|
---
|
|
77
78
|
|
|
@@ -81,15 +82,17 @@ Harness should coordinate skills through small, composable patterns.
|
|
|
81
82
|
|
|
82
83
|
### Router Decision
|
|
83
84
|
|
|
84
|
-
`harness-everything/scripts/kernel-router.js` is the public invariant-first runtime entry point. It delegates classification and guide discovery to `tier-router.js`, then returns the recommended tier, rationale, required invariants, and
|
|
85
|
+
`harness-everything/scripts/kernel-router.js` is the public invariant-first runtime entry point. It delegates classification and guide discovery to `tier-router.js`, then returns the recommended tier, rationale, required invariants, and skill suggestions. Suggestions use a **mandatory evaluation / advisory execution** contract: before omission, read the complete suggested `SKILL.md` entry and evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules. It intentionally does not emit a mandatory global execution pipeline.
|
|
86
|
+
|
|
87
|
+
A name, frontmatter description, router summary, tier label, or “routine task” judgement is not enough to skip a suggestion. Using one suggestion does not waive read-before-skip for the others. If a suggestion cannot be resolved/read, preserve `unresolved/unavailable` rather than silently converting it into “not applicable.”
|
|
85
88
|
|
|
86
89
|
### Checklist State
|
|
87
90
|
|
|
88
|
-
`todo-driven-workflow` may own explicit task decomposition state when that helps a Tier 2 or Tier 3 task. It is a
|
|
91
|
+
`todo-driven-workflow` may own explicit task decomposition state when that helps a Tier 2 or Tier 3 task. It is not a universal prerequisite. When the router suggests it, however, the agent must evaluate its actual skill flow before deciding that native host TODO tracking, a Markdown checklist, or no explicit checklist is the better choice.
|
|
89
92
|
|
|
90
93
|
### Verification Gate
|
|
91
94
|
|
|
92
|
-
`verification-loop` and `verify-gate.js` provide reusable verification mechanisms. Hosts with a compatible stop/completion hook can invoke verification mechanically; instruction-only hosts can call the same verifier explicitly before final delivery.
|
|
95
|
+
`verification-loop` and `verify-gate.js` provide reusable verification mechanisms. Hosts with a compatible stop/completion hook can invoke verification mechanically; instruction-only hosts can call the same verifier explicitly before final delivery. If `verification-loop` is router-suggested, read/evaluate its flow before omission even though the kernel already carries a verify-before-claim invariant.
|
|
93
96
|
|
|
94
97
|
### Recovery Gate
|
|
95
98
|
|
|
@@ -97,7 +100,7 @@ Harness should coordinate skills through small, composable patterns.
|
|
|
97
100
|
|
|
98
101
|
### Skill Suggestion
|
|
99
102
|
|
|
100
|
-
`self-evolve` and generated skills feed the router through manifest metadata.
|
|
103
|
+
`self-evolve` and generated skills feed the router through manifest metadata. Once the router surfaces a likely skill, the suggestion becomes an evaluation obligation: inspect its real entry/basic flow before deciding whether it applies. This does **not** make execution mandatory and does not create a fixed pipeline.
|
|
101
104
|
|
|
102
105
|
---
|
|
103
106
|
|
|
@@ -107,23 +110,23 @@ Harness should use the strongest mechanism each installation surface actually pr
|
|
|
107
110
|
|
|
108
111
|
| Platform class | Preferred coordination style | Enforcement level |
|
|
109
112
|
| --- | --- | --- |
|
|
110
|
-
| Hook/plugin-capable surfaces | Hooks/plugins run routers, guards, trackers, or completion gates automatically | Mechanical injection/blocking is possible for
|
|
111
|
-
| CLI/tool-capable but hook-less surfaces | The model explicitly calls Harness scripts at decision points | Mechanism-guided, not automatically enforced |
|
|
112
|
-
| Prompt-only surfaces | Advisory instructions include command snippets and decision rules | Self-regulated by the model |
|
|
113
|
+
| Hook/plugin-capable surfaces | Hooks/plugins run routers, guards, trackers, or completion gates automatically; prompt routing can inject the read-before-skip contract | Mechanical injection/blocking is possible for packaged mechanisms; actual model compliance with skill evaluation still needs live behavioral evidence |
|
|
114
|
+
| CLI/tool-capable but hook-less surfaces | The model explicitly calls Harness scripts at decision points and reads/evaluates suggested skills | Mechanism-guided/instruction-governed, not automatically enforced |
|
|
115
|
+
| Prompt-only surfaces | Advisory instructions include command snippets and read-before-skip decision rules | Self-regulated by the model |
|
|
113
116
|
|
|
114
117
|
One host may expose more than one surface. Codex is the important example: the general `--codex` installer path is instruction/advisory oriented, while the **local OpenAI plugin** packages session, prompt, supported-tool, subagent, and stop hooks for mechanism-tested runtime enforcement. The public OpenAI **Skills-only** submission does not include those local lifecycle hooks.
|
|
115
118
|
|
|
116
|
-
OpenCode is plugin-capable and has
|
|
119
|
+
OpenCode is plugin-capable and has partial live-host evidence for project-scope loading/state effects, but the read-before-skip behavioral contract still requires separate retained evidence before being called live-enforced. Claude Code has the broadest currently verified Harness lifecycle-hook surface.
|
|
117
120
|
|
|
118
121
|
---
|
|
119
122
|
|
|
120
123
|
## Guidance for Instruction-Only / Hook-Less Paths
|
|
121
124
|
|
|
122
|
-
Do not recreate hooks with a rigid universal handoff template. Instead, make mechanisms easy to call.
|
|
125
|
+
Do not recreate hooks with a rigid universal handoff template. Instead, make mechanisms easy to call and make the evaluation requirement explicit.
|
|
123
126
|
|
|
124
127
|
Implemented today, both wired into `bin/cli.js` and referenced by name in the advisory text each platform's installer writes (`scripts/lib/advisory-text.js`):
|
|
125
128
|
|
|
126
|
-
- `npx github:dyphn1/Harness-everything next "<prompt>"`: runs the Harness routing path and prints the recommended tier, invariant contract, and matching knowledge/skill suggestions.
|
|
129
|
+
- `npx github:dyphn1/Harness-everything next "<prompt>"`: runs the Harness routing path and prints the recommended tier, invariant contract, and matching knowledge/skill suggestions. Every suggested skill must then be read/evaluated before omission.
|
|
127
130
|
- `npx github:dyphn1/Harness-everything verify`: wraps `verify-gate.js`, running the target project's own lint/test scripts and exiting non-zero on failure. This is the explicit hook-less verification path; it is not equivalent to claiming that a host automatically runs a `Stop` hook.
|
|
128
131
|
|
|
129
132
|
Both resolve their target scripts relative to the CLI's own package install, not the caller's cwd, so they work the same way regardless of which platform-specific directory (`.codex/skills/`, `.cursor/skills/`, `.github/skills/`, ...) a copy of the `harness-everything` skill also happens to be sitting in — a fixed relative path like `harness-everything/scripts/tier-router.js` would silently point at nothing depending on which platform installed it.
|
|
@@ -146,6 +149,7 @@ Harness should not become:
|
|
|
146
149
|
- A command-only framework where skills cannot be understood by reading `SKILL.md`.
|
|
147
150
|
- A hidden daemon that performs broad actions without visible evidence.
|
|
148
151
|
- A platform-specific product that only works well when Claude Code hooks are available.
|
|
152
|
+
- A recommendation system whose suggestions are routinely ignored without inspecting the workflows they represent.
|
|
149
153
|
|
|
150
154
|
The mesh should remain local, legible, and cooperative.
|
|
151
155
|
|
|
@@ -156,10 +160,11 @@ The mesh should remain local, legible, and cooperative.
|
|
|
156
160
|
When adding or revising a skill:
|
|
157
161
|
|
|
158
162
|
1. Keep the skill independently executable.
|
|
159
|
-
2.
|
|
160
|
-
3.
|
|
161
|
-
4.
|
|
162
|
-
5.
|
|
163
|
-
6.
|
|
164
|
-
|
|
165
|
-
|
|
163
|
+
2. Put enough applicability and basic-flow information in `SKILL.md` for read-before-skip evaluation; do not hide the decision-critical contract only in optional deep references.
|
|
164
|
+
3. Add a script only when a mechanism can make the behavior more reliable than prose.
|
|
165
|
+
4. Keep script output compact and stable enough for models to react to.
|
|
166
|
+
5. Prefer gates at natural decision points: before editing, after repeated failure, before final delivery, and before persistence.
|
|
167
|
+
6. Let the model choose wording and presentation unless the skill truly needs a specific artifact format.
|
|
168
|
+
7. Document neighboring skills as recommendations, not hidden execution dependencies, unless the dependency is required for safety. Router-surfaced recommendations still inherit mandatory evaluation before omission.
|
|
169
|
+
|
|
170
|
+
The result should feel like an engineering co-pilot with good reflexes: autonomous where creativity matters, constrained where repeated mistakes are expensive, and explicit whenever evidence or applicability evaluation is needed.
|
package/docs/philosophy.md
CHANGED
|
@@ -12,12 +12,15 @@ The current mechanism/enforcement boundary is platform-specific; see [Platform C
|
|
|
12
12
|
|
|
13
13
|
Many AI development frameworks attempt to fully control the model's execution flow. They use rigid DAGs, forced chains, or complex state machines. While this works for trivial, repetitive tasks, it can suppress useful model judgment on complex, novel engineering work.
|
|
14
14
|
|
|
15
|
-
Harness takes
|
|
15
|
+
Harness takes a different approach: preserve execution autonomy, but make important decisions informed rather than superficial.
|
|
16
16
|
|
|
17
17
|
1. **Model Autonomy is Paramount:** The model should have freedom to explore, choose useful skills/tools, and design its solution inside a small set of cross-cutting invariants.
|
|
18
|
-
2. **
|
|
19
|
-
3. **
|
|
20
|
-
4. **
|
|
18
|
+
2. **Mandatory Evaluation, Advisory Execution:** When the router suggests a skill, the model must read/evaluate that skill's actual `SKILL.md` entry/basic flow before omitting it. Execution is still optional after evaluation. This prevents “advisory” from degenerating into “always ignored” without creating a fixed pipeline.
|
|
19
|
+
3. **Context-Driven Guidance:** Harness uses high-context, low-friction signals — skills, rules, local checks, and hooks/plugins where the host supports and packages them — rather than one rigid global execution path.
|
|
20
|
+
4. **Reactive Guardrails:** On integration surfaces that expose the required lifecycle/tool hooks, Harness can stay quiet until a boundary is violated or a repeated failure is detected. On advisory surfaces, the same principles remain guidance rather than automatic blocking.
|
|
21
|
+
5. **Behavior-First, Not Prompt-First:** Harness is not a collection of magic prompts. Its skills remain independently useful, while supported runtime adapters can react to real tool/session outcomes and workspace conditions.
|
|
22
|
+
|
|
23
|
+
The read-before-skip rule is intentionally narrow. It does not require every optional reference linked by a skill to be loaded, and it does not mean every suggestion must run. The agent reads the complete skill entry, evaluates `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules, follows any explicitly required applicability reference, and then decides whether execution adds value.
|
|
21
24
|
|
|
22
25
|
---
|
|
23
26
|
|
|
@@ -41,7 +44,7 @@ The Harness recovery invariant is to stop same-signature micro-retries and re-pl
|
|
|
41
44
|
|
|
42
45
|
As sessions grow, models can read too broadly or produce oversized output, degrading active context.
|
|
43
46
|
|
|
44
|
-
Harness uses narrow reads, explicit scope, compact evidence, and — where a compatible runtime adapter exists — boundary mechanisms that can react mechanically. Skill-only surfaces still receive the discipline, but not an imaginary background daemon or hook.
|
|
47
|
+
Harness uses narrow reads, explicit scope, compact evidence, and — where a compatible runtime adapter exists — boundary mechanisms that can react mechanically. Read-before-skip is bounded to the suggested skill entry and any applicability-critical material that entry explicitly requires; it does not justify indiscriminately loading the whole skills tree. Skill-only surfaces still receive the discipline, but not an imaginary background daemon or hook.
|
|
45
48
|
|
|
46
49
|
### 4. Self-Evolution (Continuous Workspace Memory)
|
|
47
50
|
|