harness-everything 0.16.1 → 0.16.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +6 -0
- package/behavioral-evals/PAIRED-BENCHMARK.md +3 -3
- package/behavioral-evals/README.md +38 -32
- package/behavioral-evals/opencode-hardlock-live.js +6 -290
- package/behavioral-evals/opencode-reflection-gate-live.js +342 -0
- package/behavioral-evals/paired-benchmark.js +3 -3
- package/ci/consistency-check.js +20 -0
- package/ci/mechanism-31-opencode-hardlock-evidence.test.js +43 -99
- package/ci/router-workflow-plan.test.js +19 -0
- package/fable-mode/SKILL.md +1 -1
- package/fable-mode/agents/fable-orchestrator.md +7 -6
- package/fable-mode/execution-guardrails/SKILL.md +1 -1
- package/fable-mode/fable-haiku/SKILL.md +1 -1
- package/fable-mode/fable-opus/SKILL.md +1 -1
- package/fable-mode/fable-sonnet/SKILL.md +1 -1
- package/harness-everything/SKILL.md +1 -1
- package/harness-everything/references/router-workflow-plan.md +2 -2
- package/harness-everything/scripts/kernel-router.js +1 -4
- package/opencode-plugin/README.md +1 -3
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +3 -2
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +7 -6
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md +2 -2
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +1 -4
- package/plugins/harness-everything/skills/zoom-out/SKILL.md +3 -3
- package/plugins/harness-everything/skills/zoom-out/references/circuit-breaker.md +4 -4
- package/zoom-out/SKILL.md +3 -3
- package/zoom-out/references/circuit-breaker.md +4 -4
|
@@ -54,14 +54,14 @@ If the structured contract is missing or invalid, the kernel emits a degraded/un
|
|
|
54
54
|
| Strategy | Selection rule | Key invariants |
|
|
55
55
|
|---|---|---|
|
|
56
56
|
| `direct-single` | Tier 1 bounded/trivial task | scope lock, verify before claim |
|
|
57
|
-
| `iterative-single` | Tier 2 ordinary diagnosis/TDD/tool loop | objective verification,
|
|
57
|
+
| `iterative-single` | Tier 2 ordinary diagnosis/TDD/tool loop | objective verification, advisory iteration guidance |
|
|
58
58
|
| `fable-staged` | Tier 3 dependent work or parallel independence not proven | stage contracts, cold verifier |
|
|
59
59
|
| `fable-parallel` | Tier 3 with explicit independent workstreams and read-only/disjoint write scope | stage contracts, parallel scope contract, synthesis barrier |
|
|
60
60
|
| `fable-multi-agent-workspace` | durable multi-session task needing reusable specialists | workspace state, handoffs, stage contracts, cold verifier |
|
|
61
61
|
|
|
62
62
|
Tier 3 alone never implies a persistent workspace. Likewise, an `independent` label alone is insufficient for `fable-parallel`: the write set must also be `read-only` or `disjoint`.
|
|
63
63
|
|
|
64
|
-
The Phase 2 `iterative-single`
|
|
64
|
+
The Phase 2 `iterative-single` plan exposes **8 iterations** as advisory planning guidance, not a machine-enforced stop. Continue when evidence justifies it; the separate rule-of-3 repeated-failure reflection gate remains the hard boundary.
|
|
65
65
|
|
|
66
66
|
## Explicit user choice and prohibitions
|
|
67
67
|
|
|
@@ -120,10 +120,7 @@ function run(raw) {
|
|
|
120
120
|
} catch (error) { failure = error; }
|
|
121
121
|
if (hostNotification && !persisted?.workflow) {
|
|
122
122
|
console.log('\n=> Host notification — no active execution contract; ignored for routing.');
|
|
123
|
-
if (failure)
|
|
124
|
-
console.error('[Workflow Router] ' + failure.message);
|
|
125
|
-
process.exitCode = 2;
|
|
126
|
-
}
|
|
123
|
+
if (failure) console.error('[Workflow Reminder] routing state was not persisted: ' + failure.message);
|
|
127
124
|
return;
|
|
128
125
|
}
|
|
129
126
|
if (result?.sanitized) console.log(result.sanitized);
|
|
@@ -62,10 +62,8 @@ normalized success/failure field to key a signature on for tools in general):
|
|
|
62
62
|
- the follow-up asks for a tokenized `zoom-out-report.md` artifact with the
|
|
63
63
|
required reflection sections; the artifact is accepted through the real
|
|
64
64
|
edit hook and records `lastReflection`
|
|
65
|
-
- the same signature failing again after that reflection was recorded
|
|
66
|
-
hard-locks the breaker
|
|
67
65
|
- after a valid reflection, the signature count resets; another three matching failures trigger another zoom-out
|
|
68
|
-
|
|
66
|
+
- `npm run harness:reset` remains an optional manual state clear; a repeated cycle does not create a permanent second-stage lock
|
|
69
67
|
|
|
70
68
|
Idle handling is bounded: after a failed verification, the plugin records that
|
|
71
69
|
the follow-up is pending and ignores repeated `session.idle` events until a new
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "harness-everything",
|
|
3
|
-
"version": "0.16.
|
|
3
|
+
"version": "0.16.2",
|
|
4
4
|
"description": "Harness OS: An Orchestrated Agent Operating System for AI-driven Development",
|
|
5
5
|
"main": "harness-everything/scripts/bootstrap.js",
|
|
6
6
|
"bin": {
|
|
@@ -40,7 +40,8 @@
|
|
|
40
40
|
"eval:behavioral": "node behavioral-evals/run.js run",
|
|
41
41
|
"eval:behavioral:validate": "node behavioral-evals/run.js validate",
|
|
42
42
|
"eval:behavioral:plugin": "node behavioral-evals/run-with-plugin.js run",
|
|
43
|
-
"eval:opencode:
|
|
43
|
+
"eval:opencode:reflection-gate-live": "node behavioral-evals/opencode-reflection-gate-live.js run",
|
|
44
|
+
"eval:opencode:hardlock-live": "node behavioral-evals/opencode-reflection-gate-live.js run",
|
|
44
45
|
"eval:paired": "node behavioral-evals/paired-benchmark.js run",
|
|
45
46
|
"eval:paired:summarize": "node behavioral-evals/paired-benchmark.js summarize",
|
|
46
47
|
"test:ab": "node ci/ab-test-harness.js validate",
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
|
|
3
3
|
"name": "harness-everything",
|
|
4
|
-
"version": "0.16.
|
|
4
|
+
"version": "0.16.2",
|
|
5
5
|
"description": "Invariant-first engineering harness for routing, verification, and failure recovery while preserving agent autonomy.",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "Miya Daniel",
|
|
@@ -53,12 +53,13 @@ overlap reject dispatch instead of becoming a verbal warning.
|
|
|
53
53
|
|
|
54
54
|
The stage map remains a living orchestration document only inside those
|
|
55
55
|
constraints. When new information invalidates it, re-plan and create revised
|
|
56
|
-
stage contracts before dispatching new work.
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
56
|
+
stage contracts before dispatching new work. `workflowPlan.limits.maxReplans`
|
|
57
|
+
is advisory orchestration guidance, not a machine-enforced counter. Re-enter
|
|
58
|
+
through `workflow-disposition.js start`; after repeated replans, consider a
|
|
59
|
+
zoom-out/reflection and keep the reason explicit, but do not block execution
|
|
60
|
+
solely because an advisory count was reached. Scope rule: deliver the task as
|
|
61
|
+
specified; new scope discovered mid-run is surfaced as a recommendation at
|
|
62
|
+
delivery, not silently built.
|
|
62
63
|
|
|
63
64
|
**2. Delegate by name and validated batch.** Every artifact-producing stage goes
|
|
64
65
|
to a named agent via the Task tool:
|
package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md
CHANGED
|
@@ -54,14 +54,14 @@ If the structured contract is missing or invalid, the kernel emits a degraded/un
|
|
|
54
54
|
| Strategy | Selection rule | Key invariants |
|
|
55
55
|
|---|---|---|
|
|
56
56
|
| `direct-single` | Tier 1 bounded/trivial task | scope lock, verify before claim |
|
|
57
|
-
| `iterative-single` | Tier 2 ordinary diagnosis/TDD/tool loop | objective verification,
|
|
57
|
+
| `iterative-single` | Tier 2 ordinary diagnosis/TDD/tool loop | objective verification, advisory iteration guidance |
|
|
58
58
|
| `fable-staged` | Tier 3 dependent work or parallel independence not proven | stage contracts, cold verifier |
|
|
59
59
|
| `fable-parallel` | Tier 3 with explicit independent workstreams and read-only/disjoint write scope | stage contracts, parallel scope contract, synthesis barrier |
|
|
60
60
|
| `fable-multi-agent-workspace` | durable multi-session task needing reusable specialists | workspace state, handoffs, stage contracts, cold verifier |
|
|
61
61
|
|
|
62
62
|
Tier 3 alone never implies a persistent workspace. Likewise, an `independent` label alone is insufficient for `fable-parallel`: the write set must also be `read-only` or `disjoint`.
|
|
63
63
|
|
|
64
|
-
The Phase 2 `iterative-single`
|
|
64
|
+
The Phase 2 `iterative-single` plan exposes **8 iterations** as advisory planning guidance, not a machine-enforced stop. Continue when evidence justifies it; the separate rule-of-3 repeated-failure reflection gate remains the hard boundary.
|
|
65
65
|
|
|
66
66
|
## Explicit user choice and prohibitions
|
|
67
67
|
|
|
@@ -120,10 +120,7 @@ function run(raw) {
|
|
|
120
120
|
} catch (error) { failure = error; }
|
|
121
121
|
if (hostNotification && !persisted?.workflow) {
|
|
122
122
|
console.log('\n=> Host notification — no active execution contract; ignored for routing.');
|
|
123
|
-
if (failure)
|
|
124
|
-
console.error('[Workflow Router] ' + failure.message);
|
|
125
|
-
process.exitCode = 2;
|
|
126
|
-
}
|
|
123
|
+
if (failure) console.error('[Workflow Reminder] routing state was not persisted: ' + failure.message);
|
|
127
124
|
return;
|
|
128
125
|
}
|
|
129
126
|
if (result?.sanitized) console.log(result.sanitized);
|
|
@@ -4,7 +4,7 @@ description: "Reflect-first circuit breaker - after 3 failures, stop edits, rebu
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.16.2
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Zoom Out (Circuit Breaker)
|
|
@@ -18,7 +18,7 @@ Stop repeated failures, rebuild the facts, and choose a safe next step.
|
|
|
18
18
|
| **Trigger / Input** | Three same-signature failures or an explicit loop/rethink request. |
|
|
19
19
|
| **Expected Output** | Fact-checked report ending in `RESUME` or `ESCALATE`. |
|
|
20
20
|
| **State Mutations** | Writes the session `zoom-out-report.md`; reset may clear breaker state. |
|
|
21
|
-
| **Enforcement Gate** | `<skills-repo-root>/hooks/scripts/rule-of-3.js` plus a valid report;
|
|
21
|
+
| **Enforcement Gate** | `<skills-repo-root>/hooks/scripts/rule-of-3.js` plus a valid report; a later three-failure cycle requests another reflection. |
|
|
22
22
|
|
|
23
23
|
## Circuit Breaker Flow
|
|
24
24
|
|
|
@@ -26,7 +26,7 @@ Stop repeated failures, rebuild the facts, and choose a safe next step.
|
|
|
26
26
|
2. Use read-only tools to restate the goal and check files, configuration, and logs.
|
|
27
27
|
3. Fill `<this-skill-dir>/templates/zoom-out-report.template.md` and end with `RESUME` or `ESCALATE`.
|
|
28
28
|
4. Resume only on an untried path; escalate genuine user decisions with options.
|
|
29
|
-
5.
|
|
29
|
+
5. If three more matching failures occur, run another zoom-out cycle. `npm run harness:reset` is only an optional manual state clear; record the insight with `self-evolve`.
|
|
30
30
|
|
|
31
31
|
## USE FOR:
|
|
32
32
|
- "you are stuck in a loop"
|
|
@@ -21,7 +21,7 @@ flowchart TD
|
|
|
21
21
|
|
|
22
22
|
Resume --> Resolved{5. Problem Cracking Succeeds?}
|
|
23
23
|
Resolved -- Yes --> SelfEvolve[Call self-evolve to Persist Insight]
|
|
24
|
-
Resolved -- No (3 More Failures) -->
|
|
24
|
+
Resolved -- No (3 More Matching Failures) --> CeaseFire
|
|
25
25
|
```
|
|
26
26
|
|
|
27
27
|
## 1. Triggers
|
|
@@ -50,7 +50,7 @@ A complete report automatically releases the circuit breaker on hook-enabled sys
|
|
|
50
50
|
## 5. Phase 4 — Decision Gate: Resume or Escalate
|
|
51
51
|
|
|
52
52
|
- **RESUME**: Selected when the fresh diagnosis identifies a clear, untried path within existing authority.
|
|
53
|
-
- **ESCALATE**: Reserved for true decision points belonging to the user (conflicting requirements, destructive migration, missing credentials/access, or repeated breaker trips).
|
|
53
|
+
- **ESCALATE**: Reserved for true decision points belonging to the user (conflicting requirements, destructive migration, missing credentials/access, or repeated breaker trips that expose a genuine decision or access gap).
|
|
54
54
|
Hand the human a **decision**, not a plea:
|
|
55
55
|
|
|
56
56
|
> "Goal: [...]. I falsified paths X, Y, Z — verified facts: [...].
|
|
@@ -61,6 +61,6 @@ Hand the human a **decision**, not a plea:
|
|
|
61
61
|
Banned: "I have tried everything and failed, please help." That reports incapability, not a decision.
|
|
62
62
|
|
|
63
63
|
## 7. Recovery
|
|
64
|
-
- **Self-recovery path**: a valid report ending in `RESUME` releases the breaker — reload the appropriate execution mode (`tdd`, `fable-mode`) and execute the new diagnosis. If the SAME failure signature accumulates 3 more failures,
|
|
65
|
-
- **Human path**: after the human answers an `ESCALATE`,
|
|
64
|
+
- **Self-recovery path**: a valid report ending in `RESUME` releases the breaker — reload the appropriate execution mode (`tdd`, `fable-mode`) and execute the new diagnosis. If the SAME failure signature accumulates 3 more failures, start another zoom-out/reflection cycle.
|
|
65
|
+
- **Human path**: after the human answers an `ESCALATE`, continue under their direction. `npm run harness:reset` (or a new session / `/clear`) is an optional manual state clear for recovery or maintenance; it is not required merely because the breaker re-tripped.
|
|
66
66
|
- Either way, once the problem is ultimately cracked, feed the insight to `self-evolve`.
|
package/zoom-out/SKILL.md
CHANGED
|
@@ -4,7 +4,7 @@ description: "Reflect-first circuit breaker - after 3 failures, stop edits, rebu
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.16.2
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Zoom Out (Circuit Breaker)
|
|
@@ -18,7 +18,7 @@ Stop repeated failures, rebuild the facts, and choose a safe next step.
|
|
|
18
18
|
| **Trigger / Input** | Three same-signature failures or an explicit loop/rethink request. |
|
|
19
19
|
| **Expected Output** | Fact-checked report ending in `RESUME` or `ESCALATE`. |
|
|
20
20
|
| **State Mutations** | Writes the session `zoom-out-report.md`; reset may clear breaker state. |
|
|
21
|
-
| **Enforcement Gate** | `<skills-repo-root>/hooks/scripts/rule-of-3.js` plus a valid report;
|
|
21
|
+
| **Enforcement Gate** | `<skills-repo-root>/hooks/scripts/rule-of-3.js` plus a valid report; a later three-failure cycle requests another reflection. |
|
|
22
22
|
|
|
23
23
|
## Circuit Breaker Flow
|
|
24
24
|
|
|
@@ -26,7 +26,7 @@ Stop repeated failures, rebuild the facts, and choose a safe next step.
|
|
|
26
26
|
2. Use read-only tools to restate the goal and check files, configuration, and logs.
|
|
27
27
|
3. Fill `<this-skill-dir>/templates/zoom-out-report.template.md` and end with `RESUME` or `ESCALATE`.
|
|
28
28
|
4. Resume only on an untried path; escalate genuine user decisions with options.
|
|
29
|
-
5.
|
|
29
|
+
5. If three more matching failures occur, run another zoom-out cycle. `npm run harness:reset` is only an optional manual state clear; record the insight with `self-evolve`.
|
|
30
30
|
|
|
31
31
|
## USE FOR:
|
|
32
32
|
- "you are stuck in a loop"
|
|
@@ -21,7 +21,7 @@ flowchart TD
|
|
|
21
21
|
|
|
22
22
|
Resume --> Resolved{5. Problem Cracking Succeeds?}
|
|
23
23
|
Resolved -- Yes --> SelfEvolve[Call self-evolve to Persist Insight]
|
|
24
|
-
Resolved -- No (3 More Failures) -->
|
|
24
|
+
Resolved -- No (3 More Matching Failures) --> CeaseFire
|
|
25
25
|
```
|
|
26
26
|
|
|
27
27
|
## 1. Triggers
|
|
@@ -50,7 +50,7 @@ A complete report automatically releases the circuit breaker on hook-enabled sys
|
|
|
50
50
|
## 5. Phase 4 — Decision Gate: Resume or Escalate
|
|
51
51
|
|
|
52
52
|
- **RESUME**: Selected when the fresh diagnosis identifies a clear, untried path within existing authority.
|
|
53
|
-
- **ESCALATE**: Reserved for true decision points belonging to the user (conflicting requirements, destructive migration, missing credentials/access, or repeated breaker trips).
|
|
53
|
+
- **ESCALATE**: Reserved for true decision points belonging to the user (conflicting requirements, destructive migration, missing credentials/access, or repeated breaker trips that expose a genuine decision or access gap).
|
|
54
54
|
Hand the human a **decision**, not a plea:
|
|
55
55
|
|
|
56
56
|
> "Goal: [...]. I falsified paths X, Y, Z — verified facts: [...].
|
|
@@ -61,6 +61,6 @@ Hand the human a **decision**, not a plea:
|
|
|
61
61
|
Banned: "I have tried everything and failed, please help." That reports incapability, not a decision.
|
|
62
62
|
|
|
63
63
|
## 7. Recovery
|
|
64
|
-
- **Self-recovery path**: a valid report ending in `RESUME` releases the breaker — reload the appropriate execution mode (`tdd`, `fable-mode`) and execute the new diagnosis. If the SAME failure signature accumulates 3 more failures,
|
|
65
|
-
- **Human path**: after the human answers an `ESCALATE`,
|
|
64
|
+
- **Self-recovery path**: a valid report ending in `RESUME` releases the breaker — reload the appropriate execution mode (`tdd`, `fable-mode`) and execute the new diagnosis. If the SAME failure signature accumulates 3 more failures, start another zoom-out/reflection cycle.
|
|
65
|
+
- **Human path**: after the human answers an `ESCALATE`, continue under their direction. `npm run harness:reset` (or a new session / `/clear`) is an optional manual state clear for recovery or maintenance; it is not required merely because the breaker re-tripped.
|
|
66
66
|
- Either way, once the problem is ultimately cracked, feed the insight to `self-evolve`.
|