harness-everything 0.30.9 → 0.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/AGENTS.md +1 -1
- package/CHANGELOG.md +34 -0
- package/README.md +8 -9
- package/VERIFICATION.md +1 -1
- package/benchmarks/results/live-host/claude-2026-10-08-issue-297/README.md +27 -0
- package/benchmarks/results/live-host/codex-2026-10-13/README.md +55 -0
- package/benchmarks/results/live-host/codex-2026-10-13/observation-records.json +88 -0
- package/benchmarks/results/pr-303-review-2026-10-13/README.md +81 -0
- package/benchmarks/results/pr-303-review-2026-10-13/ci/run.json +1330 -0
- package/benchmarks/results/pr-303-review-2026-10-13/diagnostics/mechanism-35-clean-env.log +1 -0
- package/benchmarks/results/pr-303-review-2026-10-13/diagnostics/mechanism-35-inherited-flag.log +34 -0
- package/benchmarks/results/pr-303-review-2026-10-13/diagnostics/verify-gate-captured.json +46 -0
- package/benchmarks/results/pr-303-review-2026-10-13/diagnostics/verify-gate-diagnosis.md +13 -0
- package/benchmarks/results/pr-303-review-2026-10-13/diagnostics/verify-gate.json +46 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/summary.json +57 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-collision.log +8 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-consistency.log +624 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-docs-capabilities.log +131 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-platform-compatibility.log +9 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-plugin-openai.log +6 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-plugin-submission.log +5 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-references.log +6 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-release.log +73 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-repo-contract.log +5 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-routing-invariants.log +785 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/contracts/test-routing-skills.log +44 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/final-fast/summary.json +12 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/final-fast/test-consistency.log +624 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/final-fast/test-release.log +73 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/heavy/summary.json +79 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/heavy/test-mechanism.log +1737 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/heavy/test-mutations.log +16 -0
- package/benchmarks/results/pr-303-review-2026-10-13/gates/heavy/test.log +3297 -0
- package/benchmarks/results/pr-303-review-2026-10-13/manifest.json +319 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/binding-resolution-green.log +36 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/determinism.json +174 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/quality-evidence.json +314 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/quality-report.json +638 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/red.log +240 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/untyped-fable-probe.json +22 -0
- package/benchmarks/results/pr-303-review-2026-10-13/regression/workflow-obligations-green.log +254 -0
- package/benchmarks/results/pr-303-review-2026-10-13/verification/mechanism-35-clean-env-fixed.log +1 -0
- package/benchmarks/results/pr-303-review-2026-10-13/verification/mechanism-35-inherited-flag-fixed.log +1 -0
- package/benchmarks/results/pr-303-review-2026-10-13/verification/nested-test.meta.json +8 -0
- package/benchmarks/results/pr-303-review-2026-10-13/verification/nested-test.stderr.log +90 -0
- package/benchmarks/results/pr-303-review-2026-10-13/verification/nested-test.stdout.log +3186 -0
- package/benchmarks/results/pr-303-review-2026-10-13/verification/verify-gate-final.json +46 -0
- package/bin/cli.js +1 -1
- package/ci/doc-capability-consistency.test.js +11 -1
- package/ci/invariant-routing.test.js +7 -4
- package/ci/mechanism-2ag-router-worktree-contract.test.js +2 -2
- package/ci/mechanism-2ai-workspace-fingerprint-batch.test.js +2 -1
- package/ci/mechanism-2al-codex-hooks-compat.test.js +19 -0
- package/ci/mechanism-2am-observations-session-import.test.js +221 -0
- package/ci/mechanism-2an-workflow-obligations.test.js +352 -3
- package/ci/mechanism-2ao-plugin-sync.test.js +129 -4
- package/ci/mechanism-2aq-fable-stage-bindings.test.js +168 -0
- package/ci/mechanism-2ar-binding-resolution.test.js +211 -0
- package/ci/mechanism-2w-platform-compatibility.test.js +12 -0
- package/ci/mechanism-34-observations.test.js +64 -0
- package/ci/mechanism-35-verification-contract.test.js +1 -1
- package/ci/mutation-check.js +2 -0
- package/ci/mutation-loader.js +4 -0
- package/ci/openai-plugin-package.test.js +7 -5
- package/ci/prompt-contract-regressions.test.js +7 -6
- package/ci/router-workflow-plan.test.js +109 -14
- package/ci/skill-routing-check.js +44 -15
- package/ci/workflow-plan-consumers.test.js +41 -1
- package/docs/architecture.md +10 -10
- package/docs/enforcement-locks.md +1 -1
- package/docs/mechanism-first-skill-mesh.md +12 -11
- package/docs/openai-plugin.md +1 -1
- package/docs/philosophy.md +7 -6
- package/docs/platform-capabilities.md +14 -12
- package/docs/platform-compatibility.json +5 -5
- package/docs/routing.md +16 -17
- package/docs/system-one-observations.md +36 -9
- package/docs/system-one-routing.md +10 -7
- package/docs/system-one-suggestion-gates.md +5 -5
- package/docs/workflow-runtime.md +6 -2
- package/docs/workflows/harness-everything.md +14 -14
- package/evals/harness-everything/tasks/positive-tier1.yaml +1 -1
- package/evals/harness-everything/tasks/positive-tier2.yaml +1 -1
- package/evals/harness-everything/tasks/positive-tier3.yaml +1 -1
- package/fable-mode/CONTRACT-FORMAT.md +4 -1
- package/fable-mode/SKILL.md +6 -8
- package/fable-mode/agents/fable-orchestrator.md +10 -3
- package/fable-mode/execution-guardrails/SKILL.md +1 -1
- package/fable-mode/fable-haiku/SKILL.md +1 -1
- package/fable-mode/fable-opus/SKILL.md +1 -1
- package/fable-mode/fable-sonnet/SKILL.md +1 -1
- package/fable-mode/scripts/workflow-plan-consumer.js +101 -2
- package/harness-everything/SKILL.md +8 -7
- package/harness-everything/references/router-workflow-plan.md +11 -6
- package/harness-everything/references/triage-and-tiers.md +21 -23
- package/harness-everything/schemas/router-workflow-plan.schema.json +2 -0
- package/harness-everything/scripts/kernel-router-core.js +52 -55
- package/harness-everything/scripts/kernel-router.js +48 -5
- package/harness-everything/scripts/router-contract-core.js +7 -7
- package/harness-everything/scripts/router-contract.js +1 -3
- package/harness-everything/scripts/routing-keywords.json +417 -121
- package/harness-everything/scripts/tier-router.js +64 -113
- package/hooks/scripts/contract-test.js +29 -3
- package/hooks/scripts/lib/binding-resolver.js +76 -0
- package/hooks/scripts/lib/fable-contracts.js +1 -1
- package/hooks/scripts/lib/observations.js +88 -12
- package/hooks/scripts/lib/workflow-obligations.js +292 -7
- package/hooks/scripts/lib/workflow-runtime.js +8 -1
- package/hooks/scripts/workflow-disposition.js +116 -2
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/hooks/scripts/contract-test.js +29 -3
- package/plugins/harness-everything/hooks/scripts/lib/binding-resolver.js +76 -0
- package/plugins/harness-everything/hooks/scripts/lib/fable-contracts.js +1 -1
- package/plugins/harness-everything/hooks/scripts/lib/observations.js +88 -12
- package/plugins/harness-everything/hooks/scripts/lib/workflow-obligations.js +292 -7
- package/plugins/harness-everything/hooks/scripts/lib/workflow-runtime.js +8 -1
- package/plugins/harness-everything/hooks/scripts/workflow-disposition.js +116 -2
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/scripts/lib/advisory-text.js +2 -2
- package/plugins/harness-everything/scripts/lib/generated-skills.js +78 -0
- package/plugins/harness-everything/skills/fable-mode/CONTRACT-FORMAT.md +4 -1
- package/plugins/harness-everything/skills/fable-mode/SKILL.md +6 -8
- package/plugins/harness-everything/skills/fable-mode/agents/fable-orchestrator.md +10 -3
- package/plugins/harness-everything/skills/fable-mode/execution-guardrails/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-haiku/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-opus/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/fable-sonnet/SKILL.md +1 -1
- package/plugins/harness-everything/skills/fable-mode/scripts/workflow-plan-consumer.js +101 -2
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +8 -7
- package/plugins/harness-everything/skills/harness-everything/references/router-workflow-plan.md +11 -6
- package/plugins/harness-everything/skills/harness-everything/references/triage-and-tiers.md +21 -23
- package/plugins/harness-everything/skills/harness-everything/schemas/router-workflow-plan.schema.json +2 -0
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router-core.js +52 -55
- package/plugins/harness-everything/skills/harness-everything/scripts/kernel-router.js +48 -5
- package/plugins/harness-everything/skills/harness-everything/scripts/router-contract-core.js +7 -7
- package/plugins/harness-everything/skills/harness-everything/scripts/router-contract.js +1 -3
- package/plugins/harness-everything/skills/harness-everything/scripts/routing-keywords.json +417 -121
- package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +64 -113
- package/sbom.cdx.json +7 -7
- package/scripts/codex-hooks-compat.js +39 -14
- package/scripts/lib/advisory-text.js +2 -2
- package/scripts/lib/generated-skills.js +78 -0
- package/scripts/plugin-sync.js +100 -29
- package/scripts/system-one-observations-export.js +29 -2
- package/scripts/system-one-observations-import-codex-sessions.js +307 -0
- package/scripts/system-one-observations-review.js +11 -1
|
@@ -15,19 +15,21 @@ Statuses are intentionally explicit:
|
|
|
15
15
|
- `Unsupported`: official host documentation explicitly rules out the capability.
|
|
16
16
|
- `Unknown`: the capability or its current host behavior was not established.
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
Codex and OpenCode are `Partial` in `liveHostVerification`. The Codex artifact verifies user-hook compatibility capture for two consecutive turns on one Windows Codex CLI host; native plugin-hook mounting and other host/version combinations remain unverified. OpenCode's retained evidence supports project-scope `.js` loading and edit/verification state on OpenCode 1.18.31 (macOS), not durable hard enforcement. No fresh artifacts exist yet for the other surfaces, managed-workspace import, or public OpenAI approval.
|
|
19
19
|
|
|
20
20
|
### Routing contract vs. host evidence
|
|
21
21
|
|
|
22
|
-
The
|
|
22
|
+
The workflow plan controller rejects present non-array required/optional binding declarations before recording a requirement plan. This input-validation contract is covered by deterministic mechanism tests and does not change live-host evidence status.
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
The repository routing contract uses **step-scoped workflow selection**. Normalized keyword/domain signals help compose ordered requirements but never select or load document paths. Each step/stage declares its required and optional skill/reference bindings; only active bindings are disclosed, and required bindings resolve before the step/stage passes. Unknown or unavailable required bindings stay visible and unresolved; missing evidence produces reminders rather than a persistent workflow lock.
|
|
25
|
+
|
|
26
|
+
This is a **repository/agent contract**, not an automatic upgrade to any platform row below. A hook can prove that the contract was injected and record an agent's binding disposition; neither proves that a live host actually loaded or followed the referenced content. Instruction-only surfaces can carry the same rule without mechanically enforcing it. Promoting behavior to live-host compliance requires retained host/session evidence under the #82 compatibility program. Claude and local OpenAI packages carry workflow entry/completion **reminders** plus separate Rule-of-3/permission boundaries, but this package/mechanism change does not upgrade `platform-compatibility.json` live-host status values. See [workflow runtime](workflow-runtime.md) for scope, worktree isolation, and failure behavior. OpenCode and instruction-only/public Skills-only surfaces do not acquire equivalent runtime gating. Shell inspection is not a filesystem sandbox; absent hooks, agent-writable state, and indirect script effects remain limits.
|
|
25
27
|
|
|
26
28
|
## Current capability summary
|
|
27
29
|
|
|
28
30
|
| Platform surface | Standalone skills | Plugin support | Hooks/lifecycle | Project scope | Global scope | Current evidence boundary |
|
|
29
31
|
| --- | --- | --- | --- | --- | --- | --- |
|
|
30
|
-
| Codex | `Supported` | `Mechanism verified` | `
|
|
32
|
+
| Codex | `Supported` | `Mechanism verified` | `Partial`: user-hook compatibility capture is live-verified for `UserPromptSubmit` and `Stop` on one Windows Codex CLI host; native plugin hooks remain mechanism-verified | `.agents/skills/`, `AGENTS.md` | `~/.agents/skills/` | [Live evidence](../benchmarks/results/live-host/codex-2026-10-13/README.md) verifies two consecutive turns on Codex CLI 0.162.1; native plugin mounting, other lifecycle events, and other versions remain unverified. |
|
|
31
33
|
| ChatGPT / OpenAI Plugin | `Unknown` for a repository-local standalone path | `Mechanism verified` | `Partial`: local Work/Codex hooks are separate from ordinary Chat | Managed workspace marketplace | Not established | The public Skills-only bundle is mechanically reproducible; import, approval, and live behavior remain `Unknown`. |
|
|
32
34
|
| Claude Code | `Supported` | `Supported` | `Mechanism verified` for Harness configuration | `.claude/skills/` | `~/.claude/skills/` | Native Claude surfaces and the state-specific plugin sync command are mechanically checked; fresh-host execution is not preserved here. |
|
|
33
35
|
| OpenCode | `Supported` | `Partial` live-host loading for the project-scope, `.js`-filename install path on OpenCode 1.18.31 (macOS); other install paths remain `Mechanism verified` | `Partial` live-host evidence for project-scope `.js` loading and edit/verification state; hard lock reported only, with no retained blocked-tool trace and a post-reset final snapshot | `.opencode/skills/`, `.agents/skills/`, `.opencode/plugins/` | `~/.config/opencode/skills/`, `~/.agents/skills/`, `~/.config/opencode/plugins/` | [Retained evidence](../benchmarks/results/live-host/opencode-2026-09-16/README.md): loading and state effects verified in one host session; final snapshot post-reset; hard lock only an interactive observation; reflection operator-seeded, then agent-rewritten; no raw transcript or behavioral-effectiveness claim. `.mjs` auto-discovery broken (issue #127); global scope, npm-package installation, and other host versions remain unverified. |
|
|
@@ -61,16 +63,16 @@ The general installer’s target paths are tested independently from host discov
|
|
|
61
63
|
|
|
62
64
|
These targets apply to both default link mode and explicit `--copy` mode. A canonical store is only a deduplication detail; it cannot make an unsupported host path supported. Install/uninstall tests also protect user-owned files and cover Linux, Windows, and macOS round-trip fixtures.
|
|
63
65
|
|
|
64
|
-
The advisory installer text generated for Codex/Cursor/Copilot/Continue/Hermes carries the same
|
|
66
|
+
The advisory installer text generated for Codex/Cursor/Copilot/Continue/Hermes carries the same step-scoped binding rule. That proves only what Harness writes, not that the host/model loaded or followed an active binding.
|
|
65
67
|
|
|
66
68
|
## Codex / local OpenAI plugin boundary
|
|
67
69
|
|
|
68
70
|
Codex has two paths that must not be collapsed:
|
|
69
71
|
|
|
70
|
-
1. The general `--codex` installer writes advisory `AGENTS.md` plus repo-scoped skills under `.agents/skills/`. Its generated instructions require
|
|
71
|
-
2. The local OpenAI plugin at `plugins/harness-everything/` packages the skills plus lifecycle hooks for session start, prompt routing, supported local tool calls, subagent lifecycle, and stop verification. Those hooks are mechanically checked
|
|
72
|
+
1. The general `--codex` installer writes advisory `AGENTS.md` plus repo-scoped skills under `.agents/skills/`. Its generated instructions require resolving bindings declared for the active step, but this remains instruction-governed.
|
|
73
|
+
2. The local OpenAI plugin at `plugins/harness-everything/` packages the skills plus lifecycle hooks for session start, prompt routing, supported local tool calls, subagent lifecycle, and stop verification. Those hooks are mechanically checked. A separate live run verifies the user-hook compatibility fallback for prompt/stop observation only; native plugin hook mounting remains unverified.
|
|
72
74
|
|
|
73
|
-
The package
|
|
75
|
+
The package remains subject to the host’s hook review/trust flow. The retained Codex CLI session proves that the selected user-level observation hooks were trusted and fired on one Windows setup; it does not prove that the model loaded or followed an active binding or that another ChatGPT/Codex installation loaded the same hooks.
|
|
74
76
|
|
|
75
77
|
### Native plugin install/update synchronization
|
|
76
78
|
|
|
@@ -98,13 +100,13 @@ The command detects Claude Code and Codex independently, ensures the Harness mar
|
|
|
98
100
|
|
|
99
101
|
Mutating commands are issued with the current option set and retried without any option the host rejects, covering both commander wording (`unknown option '--json'`) and clap wording (`unexpected argument '--json' found`), so older Claude and Codex builds stay supported.
|
|
100
102
|
|
|
101
|
-
The installed branch is intentionally an update branch even when the current release is already the newest version; the host command may report “already latest.” Codex marketplace upgrade is the current native refresh operation; the
|
|
103
|
+
The installed branch is intentionally an update branch even when the current release is already the newest version; the host command may report “already latest.” Codex marketplace upgrade is the current native refresh operation. After successful Codex plugin install/update and verification, `plugin-sync.js` reads the installed plugin's tagged JSON source. It refreshes compatibility hooks only for `source: "local"` with an absolute filesystem path; `git`, `git-subdir`, `npm`, `remote`, unknown or missing variants, and relative paths fail closed before the installer runs. It does not reinterpret a repository subdirectory relative to the sync process working directory. An ownership/configuration error also makes Codex sync report failure. The installer does not alter trust state or private `config.toml`. Deterministic tests cover this source-shape policy; live Codex marketplace source reporting/sync remains unverified. The separate compatibility fallback has one retained live dispatch artifact on Windows, as recorded in the matrix.
|
|
102
104
|
|
|
103
105
|
On Windows, the synchronizer resolves `codex`/`claude` the same way an interactive shell does: it walks `PATH` directories in order and tries `PATHEXT` extensions within each directory before moving on, so directory order — not which candidate happens to be a directly launchable `.exe` — decides the match. This matters because Node's non-shell `spawnSync` cannot execute a `.cmd`/`.bat` shim directly and, left to its own resolution, can silently prefer an unrelated, older standalone `.exe` elsewhere on `PATH` over a newer npm-installed shim with no error to signal it. Only the one case Windows genuinely requires an interpreter for — a resolved `.cmd`/`.bat` target — is run through `ComSpec`, using the already-resolved absolute path rather than re-resolving the bare command name.
|
|
104
106
|
|
|
105
107
|
### Explicit Codex user-hook compatibility fallback
|
|
106
108
|
|
|
107
|
-
Current Codex documentation supports
|
|
109
|
+
Current Codex documentation supports lifecycle hooks for manually installed Codex desktop plugins and requires users to review and trust non-managed hook definitions. The retained codex-cli 0.157.1 evidence records plugin_hooks as removed, so the user-hook compatibility installer remains necessary for that CLI path. Harness provides the compatibility path alongside the native plugin package:
|
|
108
110
|
|
|
109
111
|
```bash
|
|
110
112
|
harness codex-hooks install
|
|
@@ -112,11 +114,11 @@ harness codex-hooks status
|
|
|
112
114
|
harness codex-hooks uninstall
|
|
113
115
|
```
|
|
114
116
|
|
|
115
|
-
The
|
|
117
|
+
The installer copies the current packaged runtime into `$CODEX_HOME/harness-everything/compat-hooks/` and merges absolute-path entries into `$CODEX_HOME/hooks.json`. harness codex-hooks status reports `inactive` when the compatibility hooks are absent, or `configured-unverified` when their files are healthy; it cannot read Codex trust state. A successful Codex run of plugin-sync.js installs or refreshes this copy automatically. The installer does **not** modify `config.toml`, trust hashes, approval policy, or hook enablement. Codex must still show/review/trust the resulting non-managed hooks through its normal `/hooks` flow.
|
|
116
118
|
|
|
117
119
|
Ownership is deliberately strict: update/uninstall removes only exact entries recorded in the Harness compatibility manifest. If an installed Harness entry was edited after installation, the operation fails closed rather than overwriting or deleting that user-modified entry. Unrelated pre-existing and later-added user hooks are preserved.
|
|
118
120
|
|
|
119
|
-
|
|
121
|
+
The compatibility path registers hook commands in Codex user configuration and covers the missing plugin-mount route where user hooks dispatch. It does not repair a Codex carrier that fails to dispatch `PreToolUse` at all; those host-level execution-path limitations remain a live compatibility boundary.
|
|
120
122
|
|
|
121
123
|
The public OpenAI Skills-only plugin is narrower again. Its ZIP contains the packaged `skills/` tree and referenced files, but not the local `.codex-plugin` lifecycle hooks. Public submission readiness is therefore a package/form contract, not evidence of public-directory approval or live hook execution.
|
|
122
124
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"format": "harness-platform-compatibility-v1",
|
|
3
|
-
"last_reviewed": "2026-
|
|
3
|
+
"last_reviewed": "2026-10-13",
|
|
4
4
|
"status_vocabulary": {
|
|
5
5
|
"Supported": "An official or native mechanism exists and the Harness behavior is verified.",
|
|
6
6
|
"Mechanism verified": "Harness package, adapter, or installer checks pass, but a real host run is not confirmed.",
|
|
@@ -109,13 +109,13 @@
|
|
|
109
109
|
"standaloneSkills": {"status": "Supported", "paths": {"project": [".agents/skills"], "global": ["~/.agents/skills"]}},
|
|
110
110
|
"pluginSupport": {"status": "Mechanism verified", "paths": {"manifest": ["plugin.json", ".codex-plugin/plugin.json"]}},
|
|
111
111
|
"bundledSkills": {"status": "Mechanism verified", "notes": "The local package contains a root skills/ tree."},
|
|
112
|
-
"hooksLifecycle": {"status": "
|
|
112
|
+
"hooksLifecycle": {"status": "Partial", "notes": "Codex documents plugin-bundled hooks for manually installed desktop plugins, and the local package is mechanism-tested. The plugin intentionally does not declare unsupported PostToolUseFailure; Codex uses documented PostToolUse semantics instead. Harness also provides a user-config compatibility installer that copies the packaged runtime and merges exact-owned entries into $CODEX_HOME/hooks.json without changing trust/config.toml; successful plugin sync refreshes this copy. Live evidence verifies the UserPromptSubmit and Stop observation handlers through the compatibility fallback on one Windows Codex CLI 0.162.1 host; native plugin-bundled hook mounting and other lifecycle events remain unverified."},
|
|
113
113
|
"projectScope": {"status": "Mechanism verified", "paths": {"skills": [".agents/skills"], "instructions": ["AGENTS.md"]}},
|
|
114
114
|
"globalScope": {"status": "Mechanism verified", "paths": {"skills": ["~/.agents/skills"]}},
|
|
115
|
-
"installUninstallSymmetry": {"status": "Mechanism verified", "notes": "General Harness installer ownership is mechanism-tested. The optional codex-hooks compatibility fallback separately has exact-entry update/uninstall ownership tests and preserves unrelated user hooks
|
|
115
|
+
"installUninstallSymmetry": {"status": "Mechanism verified", "notes": "General Harness installer ownership is mechanism-tested. The optional codex-hooks compatibility fallback separately has exact-entry update/uninstall ownership tests and preserves unrelated user hooks. Live evidence confirms trusted dispatch for the two observation handlers on one Windows Codex CLI host; native plugin mounting and other scopes remain unverified."},
|
|
116
116
|
"assetsReferences": {"status": "Mechanism verified", "notes": "Packaged skill references are checked and remain inside the package."},
|
|
117
|
-
"updateSync": {"status": "Mechanism verified", "notes": "scripts/plugin-sync.js detects installed state and uses marketplace upgrade for an installed Codex plugin;
|
|
118
|
-
"liveHostVerification": {"status": "
|
|
117
|
+
"updateSync": {"status": "Mechanism verified", "notes": "scripts/plugin-sync.js detects installed state and uses marketplace upgrade for an installed Codex plugin; after successful install/update verification it refreshes the local codex-hooks compatibility runtime and hooks.json entries. Unsupported CLI options are dropped and retried, and a Codex build without codex plugin list/add is reported as an actionable capability-boundary skip. Live marketplace source reporting and plugin-sync refresh remain unverified; the separate compatibility fallback dispatch is live-verified as recorded in the evidence artifact."},
|
|
118
|
+
"liveHostVerification": {"status": "Partial", "evidence": "benchmarks/results/live-host/codex-2026-10-13/README.md", "notes": "The preserved artifact verifies two consecutive observation captures through the user-hook compatibility fallback on Codex CLI 0.162.1 for Windows, including current-turn labels. Native plugin mounting, marketplace source reporting, other lifecycle events, and other host/version combinations remain unverified."}
|
|
119
119
|
}
|
|
120
120
|
},
|
|
121
121
|
{
|
package/docs/routing.md
CHANGED
|
@@ -6,7 +6,7 @@ The current architecture follows two coupled rules:
|
|
|
6
6
|
|
|
7
7
|
> **Contract-first lifecycle; flexible reasoning/implementation and execution.**
|
|
8
8
|
>
|
|
9
|
-
> **
|
|
9
|
+
> **Knowledge is step-scoped: the active step declares required/optional bindings, and required bindings resolve before that step passes.**
|
|
10
10
|
|
|
11
11
|
A strong model remains free to decide *how* to perform the work. It may not decide that an applicable selected lifecycle can be skipped because the task feels simple or already understood.
|
|
12
12
|
|
|
@@ -17,7 +17,7 @@ For software/project work, Harness establishes routing context and lightweight r
|
|
|
17
17
|
1. **Route before execution** — establish scope/tier and the smallest justified topology.
|
|
18
18
|
2. **Verify before claim** — completion requires objective evidence appropriate to the change.
|
|
19
19
|
3. **Re-plan after repeated failure** — after three same-signature failures, stop micro-retrying and use a fresh diagnosis / `zoom-out`.
|
|
20
|
-
4. **
|
|
20
|
+
4. **Bind knowledge by step** — compose ordered requirement steps and declare their required/optional skill or reference bindings before execution.
|
|
21
21
|
5. **Resolve selected workflow** — satisfy the selected topology's required obligations before completion; tactics MAY adapt, but required lifecycle evidence cannot be silently skipped.
|
|
22
22
|
6. **Surface status** — non-trivial work MUST keep the user informed through the single Harness Status format at required phase boundaries.
|
|
23
23
|
|
|
@@ -27,10 +27,11 @@ The model owns tools, implementation technique, reasoning, and decomposition det
|
|
|
27
27
|
|
|
28
28
|
```text
|
|
29
29
|
kernel-router.js
|
|
30
|
-
├─ delegates classification /
|
|
31
|
-
├─ preserves tier + rationale +
|
|
30
|
+
├─ delegates classification / knowledge-signal detection → tier-router.js
|
|
31
|
+
├─ preserves tier + rationale + normalized knowledge signals
|
|
32
32
|
├─ emits required invariants + structured workflow plan
|
|
33
|
-
├─
|
|
33
|
+
├─ treats keyword hits as step-composition signals, not document selectors
|
|
34
|
+
├─ discloses bindings only for the active step / dependency-ready Fable stage
|
|
34
35
|
├─ records the selected workflow contract
|
|
35
36
|
└─ exposes warnings/degraded/escape evidence instead of silently hiding limitations
|
|
36
37
|
```
|
|
@@ -89,19 +90,17 @@ label appeared on 1 of 4 turns while it sat inside the Harness Status contract
|
|
|
89
90
|
(which is scoped to non-trivial work), and on 5 of 6 turns as its own final
|
|
90
91
|
section.
|
|
91
92
|
|
|
92
|
-
##
|
|
93
|
+
## Knowledge signals and active-step bindings
|
|
93
94
|
|
|
94
|
-
|
|
95
|
+
Router keywords and domain matches are normalized planning inputs. They do not select, load, or print skill/reference paths. Requirement composition assigns needed knowledge to ordered steps:
|
|
95
96
|
|
|
96
|
-
|
|
97
|
+
- Each step declares `requiredBindings` and `optionalBindings` by stable id, with a concrete path when known.
|
|
98
|
+
- Only the active single-agent step or dependency-ready Fable stage exposes its bindings. Retained prompts and notifications keep this active-step view compact; future paths stay hidden.
|
|
99
|
+
- A required binding MUST be loaded and its applicable skill core contract followed before the step can pass. Optional bindings resolve as loaded or not-needed.
|
|
100
|
+
- An unknown/unavailable required binding remains visible and unresolved; it cannot be marked loaded without a declared path and evidence.
|
|
101
|
+
- On Fable plans, the existing stage dependency graph controls readiness. A successful objective check records `binding-unresolved` until bindings resolve; rerun the exact check before the stage passes.
|
|
97
102
|
|
|
98
|
-
|
|
99
|
-
2. **MUST** evaluate `USE FOR`, `DO NOT USE FOR`, workflow/basic flow, and hard rules.
|
|
100
|
-
3. Read extra material only when the entry explicitly requires it for applicability.
|
|
101
|
-
4. **MUST** resolve the suggestion as `use`, `not-applicable`, or `unresolved/unavailable`.
|
|
102
|
-
5. If disposition is `use`, the skill's core contract **MUST** be followed. If it is `not-applicable`, a concise flow-grounded reason **MUST** be retained.
|
|
103
|
-
|
|
104
|
-
A name, description, router summary, tier label, or generic “routine/common task” judgement is not sufficient evidence for `not-applicable`.
|
|
103
|
+
For every active required skill binding, read its complete `SKILL.md` entry and follow its applicable core contract. If an active requirement does not need a discovered knowledge signal, do not bind or load an unrelated document. This replaces the former global read-every-suggestion rule while preserving per-step obligations.
|
|
105
104
|
|
|
106
105
|
For the **selected workflow topology**:
|
|
107
106
|
|
|
@@ -135,7 +134,7 @@ Typical triggers: typo/docs correction, narrow local edit, bounded explanation,
|
|
|
135
134
|
|
|
136
135
|
### Tier 2 — Standard
|
|
137
136
|
|
|
138
|
-
Typical triggers: normal feature work, bug fixes, multi-file changes, focused benchmark/review. Usually selects `iterative-single`: bounded reason/act work plus objective verification.
|
|
137
|
+
Typical triggers: normal feature work, bug fixes, multi-file changes, focused benchmark/review. Usually selects `iterative-single`: bounded reason/act work plus objective verification. Compose ordered requirements before execution; bind TDD to behavior-change steps, verification-loop to verification steps, and domain knowledge only where that step needs it.
|
|
139
138
|
|
|
140
139
|
Operation labels such as `audit`, `evaluate`, `benchmark`, or `compare` describe **what** work is being done; they do not by themselves establish macro scope. A focused audit/review remains bounded unless an independent scope/structure signal (for example repository-wide, all files/modules, or multi-workstream structure) elevates it. A structured A/B experiment is different: its explicit control/treatment comparison is itself a multi-lane execution structure, so `A/B test` / `A/B benchmark` remains a macro signal.
|
|
141
140
|
|
|
@@ -161,7 +160,7 @@ The [workflow runtime contract](workflow-runtime.md) defines the executable boun
|
|
|
161
160
|
|
|
162
161
|
- `workflow-gate.js` observes shell/direct mutation, Fable correlation, and major-workflow Git isolation and emits reminders.
|
|
163
162
|
- `workflow-stop-gate.js` reports unresolved stage/verification evidence without rejecting Stop.
|
|
164
|
-
- `workflow-disposition.js` starts/replans the run, records a scoped stage escape, or reports `blocked`.
|
|
163
|
+
- `workflow-disposition.js` starts/replans the run, resolves active step/stage bindings, records a scoped stage escape, or reports `blocked`.
|
|
165
164
|
|
|
166
165
|
Follow-up prompts retain useful evidence, but unresolved cognitive workflow state does not lock mutation or completion.
|
|
167
166
|
|
|
@@ -2,8 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
Status: **accepted**, 2026-09-27. Owner decisions are recorded below.
|
|
4
4
|
Implementation: `hooks/scripts/observation-hook.js`,
|
|
5
|
-
`hooks/scripts/lib/observations.js`, `scripts/system-one-observations-export.js`
|
|
6
|
-
|
|
5
|
+
`hooks/scripts/lib/observations.js`, `scripts/system-one-observations-export.js`, `scripts/system-one-observations-import-codex-sessions.js` and
|
|
6
|
+
`scripts/system-one-observations-review.js`; tests in
|
|
7
7
|
`ci/mechanism-34-observations.test.js`. Issue
|
|
8
8
|
[#233](https://github.com/dyphn1/Harness-everything/issues/233).
|
|
9
9
|
|
|
@@ -42,8 +42,8 @@ A turn runs from `UserPromptSubmit` to `Stop`. Its record:
|
|
|
42
42
|
| `id`, `sessionId`, `turn`, `host`, `observedAt` | hooks | `id` = hash of session id + turn |
|
|
43
43
|
| `prompt` | `UserPromptSubmit` | stored in the private text store only (see Storage) |
|
|
44
44
|
| `previous` | the last assistant message before the prompt | truncated to 2 KB; text store only |
|
|
45
|
-
| `router` | kernel-router output | lexical tier, strategy,
|
|
46
|
-
| `behavior` | `PostToolUse`, `PostToolUseFailure` | counts and categories, never arguments: files written, distinct repositories written, files created, commands by class (git, gh, test, build, package, shell), skills loaded or read, subagents started. Failed attempts are counted separately (`toolFailures`, `failedWrites`, `failedSkillLoads`) and never count as a write or a load |
|
|
45
|
+
| `router` | kernel-router output | lexical tier, strategy, normalized knowledge signals, active step/stage bindings; System One shadow scores when present |
|
|
46
|
+
| `behavior` | `PostToolUse`, `PostToolUseFailure` | counts and categories, never arguments: files written, distinct repositories written, files created, commands by class (git, gh, test, build, package, shell), skills loaded or read, subagents started. Failed attempts are counted separately (`toolFailures`, `failedWrites`, `failedSkillLoads`) and never count as a write or a load. `null` means the capture source did not observe tool events; it is not a zero count. |
|
|
47
47
|
| `workflow` | `workflow-run.json` / `workflow-disposition` | selected and confirmed strategy |
|
|
48
48
|
| `selfReport` | label line in the final message | see below; `null` if missing or invalid |
|
|
49
49
|
| `derived` | exporter | labels computed from the fields above |
|
|
@@ -98,6 +98,12 @@ purposes:
|
|
|
98
98
|
- **Future features.** The counters can train or check later models
|
|
99
99
|
without re-reading transcripts.
|
|
100
100
|
|
|
101
|
+
Session backfill reads prompts and final labels, but does not reconstruct
|
|
102
|
+
tool events or file writes. Its `behavior` is therefore `null`. Exports count
|
|
103
|
+
observed and unobserved behavior separately and derive contradictions only
|
|
104
|
+
from observed behavior. The default review page includes unobserved records
|
|
105
|
+
and identifies the evidence gap.
|
|
106
|
+
|
|
101
107
|
**Owner review.** Missing labels, contradictions, and turns where the
|
|
102
108
|
router's tier and the self-reported tier differ go to a blind review page.
|
|
103
109
|
The owner decides validity, context dependence and tier; each decision
|
|
@@ -157,11 +163,16 @@ warning sign. The review page samples the turns where they differ.
|
|
|
157
163
|
it wrote no label; the same text as JSON reached the model and all five
|
|
158
164
|
test turns carried a label. The Codex manifest therefore runs the router
|
|
159
165
|
through `hooks/scripts/codex-user-prompt.js`, which wraps its output.
|
|
160
|
-
-
|
|
161
|
-
`
|
|
162
|
-
|
|
163
|
-
|
|
166
|
+
- The retained `codex-cli 0.157.1` evidence found plugin hooks removed
|
|
167
|
+
(`plugin_hooks`); the local CLI path therefore uses the user-hook
|
|
168
|
+
compatibility installer. Successful `plugin:hosts` runs now install or
|
|
169
|
+
refresh that copied runtime and its `hooks.json` entries. The installer
|
|
170
|
+
does not change trust state; Codex still requires hook review/trust in
|
|
171
|
+
`/hooks`. Status reports `inactive` when absent and
|
|
172
|
+
`configured-unverified` when the files are healthy, because this command
|
|
173
|
+
cannot read Codex trust state.
|
|
164
174
|
- The final message comes from the rollout's `task_complete` event.
|
|
175
|
+
- Fresh live-host verification on Windows (Codex CLI 0.162.1, 2026-10-13) exercised the user-hook compatibility fallback: only the `UserPromptSubmit` and `Stop` observation hooks were individually reviewed/trusted. Two consecutive real turns produced `host: "codex"` records with distinct current `turn_id` values and their respective valid self-reports. The sanitized record excerpt and scope limits are preserved in [Codex live-host evidence](../benchmarks/results/live-host/codex-2026-10-13/README.md). This does not verify native plugin-bundled hooks, other lifecycle events, or tool behavior capture.
|
|
165
176
|
- The final message must belong to the current turn. A transcript scan
|
|
166
177
|
stops at the start of the current turn (Codex `task_started` or a user
|
|
167
178
|
message; a Claude user prompt) and a Codex `task_complete` must carry the
|
|
@@ -174,18 +185,34 @@ warning sign. The review page samples the turns where they differ.
|
|
|
174
185
|
- A missing field records `null` with a reason code. Collection must never
|
|
175
186
|
block, slow down, or fail a turn; each hook has a short time budget.
|
|
176
187
|
|
|
188
|
+
## Session backfill
|
|
189
|
+
|
|
190
|
+
To recover self-reported labels written before the observation hooks were active, run:
|
|
191
|
+
|
|
192
|
+
node scripts/system-one-observations-import-codex-sessions.js --since 2026-09-27
|
|
193
|
+
|
|
194
|
+
The importer scans local Codex rollout files and pairs each completed turn with its first user text item. It reads the final label from `task_complete.last_agent_message`; when that field is absent or not a string, the last assistant `response_item` can provide it unless that item is marked `commentary`. When a string completion has no label and the last assistant item is not marked `commentary` and has a valid label, their bodies are compared after `obs.stripLabels` removes label comments and trims surrounding whitespace. A match lets the assistant label provide the missing label; a difference reports `ambiguousFinalMessage`, skips the turn, and clears previous-message context rather than choosing one representation. An explicitly unlabeled completion never inherits a commentary or earlier assistant label, and an invalid completion label is not replaced by the assistant label.
|
|
195
|
+
|
|
196
|
+
The importer accepts only valid label lines, applies the same email/home-path redaction, and appends idempotently to the private observations store. Backfill and live hook records deduplicate by `(host, writer.sessionId, writer.turnId)`. For older live records without a turn ID, an exact turn-and-text match is skipped; a partial match is counted as ambiguous and skipped to avoid silently duplicating training data. Duplicate and ambiguous legacy live matches are checked atomically under the store lock. A bounded lock timeout is reported as a persistence failure, not as a duplicate, and gives the importer a nonzero exit status; migration lock timeouts are reported the same way. Existing backfill behavior placeholders are migrated to `null`. Reports contain counts and generic error descriptions without printing session text. The cutoff defaults to the date the label contract was accepted. The importer never uploads or writes session content into the repository.
|
|
197
|
+
|
|
177
198
|
## Export
|
|
178
199
|
|
|
179
200
|
`scripts/system-one-observations-export.js` writes to the private data
|
|
180
201
|
repository:
|
|
181
202
|
|
|
182
203
|
- `prompts-observed.jsonl`: `{id, family, source: "observation", split, text}`
|
|
183
|
-
- `labels-observed.jsonl`: `{id, validity, contextDependent, tier, tierSource, intents, workflow, skills}`
|
|
204
|
+
- `labels-observed.jsonl`: `{id, validity, contextDependent, tier, tierSource, intents, workflow, skills, behaviorEvidence, contradictions}`
|
|
184
205
|
|
|
185
206
|
`family` groups near-duplicate prompts. The split is by time: the newest 15%
|
|
186
207
|
is validation. The existing holdouts stay frozen and separate. Observed data
|
|
187
208
|
may later seed a new holdout, but only after owner review.
|
|
188
209
|
|
|
210
|
+
For a training candidate restricted to labels from the accepted label-line date, use an output directory outside the repository:
|
|
211
|
+
|
|
212
|
+
node scripts/system-one-observations-export.js --out <private-output-dir> --since 2026-09-27 --self-report-only
|
|
213
|
+
|
|
214
|
+
This keeps unlabeled observations out of the candidate and leaves existing model and holdout directories untouched.
|
|
215
|
+
|
|
189
216
|
## Phases
|
|
190
217
|
|
|
191
218
|
1. **Contract.** Define the label-line format in `AGENTS.md` rule 11 and in
|
|
@@ -360,9 +360,9 @@ small model only has to separate a few options at a time.
|
|
|
360
360
|
`discuss`, `git`, `fix`, `edit`, `feature`, `refactor`, `review`, `test`,
|
|
361
361
|
`docs`, `plan`, `investigate` or `unclassified`. A prompt has one primary
|
|
362
362
|
intent and may have secondary intents.
|
|
363
|
-
3. **
|
|
364
|
-
|
|
365
|
-
|
|
363
|
+
3. **Skill relevance**: candidate signals for requirement composition,
|
|
364
|
+
scored against canonical skill descriptions. These candidates do not select
|
|
365
|
+
paths, trigger loads, or bypass the active-step binding contract.
|
|
366
366
|
|
|
367
367
|
Each stage has its own gates, and a stage ships only when it passes them.
|
|
368
368
|
The relevance-native plan in
|
|
@@ -371,8 +371,10 @@ adds a validity stage first and runs intent before tier, because tier3 is
|
|
|
371
371
|
composed from the intent scores. Tier was trained first because the
|
|
372
372
|
reviewed holdout exists for it. Later stages reuse the
|
|
373
373
|
same collector, labeler, trainer and evaluator with a different catalog.
|
|
374
|
-
|
|
375
|
-
ownership, the Rule of 3 and
|
|
374
|
+
Relevance candidates are advisory inputs to requirement composition: explicit
|
|
375
|
+
workflow requests, action gates, memory ownership, the Rule of 3 and
|
|
376
|
+
deterministic policy always take precedence. A candidate becomes a skill path
|
|
377
|
+
only when a specific ordered requirement declares it as a binding.
|
|
376
378
|
|
|
377
379
|
## Rollout gates
|
|
378
380
|
|
|
@@ -383,8 +385,9 @@ provenance, repeatability and advisory rules below still apply.
|
|
|
383
385
|
|
|
384
386
|
Phase 4 requires a separately reviewed, family-disjoint holdout of at least 200
|
|
385
387
|
cases, at least 50 each in English and Traditional Chinese. System One output is
|
|
386
|
-
advisory: it informs
|
|
387
|
-
|
|
388
|
+
advisory: it informs requirement composition and provides skill-relevance
|
|
389
|
+
candidates. It never grants, removes policy, or loads paths. The owner therefore
|
|
390
|
+
set the target at accepted precision ≥85% (the
|
|
388
391
|
80–90% band), coverage ≥80%, and macro-F1 at least the lexical baseline. A
|
|
389
392
|
probabilistic scorer small enough to ship inside a skills plugin (a few MB, not
|
|
390
393
|
hundreds) is not expected to reach 98%.
|
|
@@ -42,12 +42,12 @@ All stages share the scoring and the calibration. Only the readout differs.
|
|
|
42
42
|
| Intent | the 12 intents of [system-one-intent.md](system-one-intent.md) | set: every intent at or above its threshold, highest `p` first, truncated at the cap | 3 |
|
|
43
43
|
| Tier | `tier1`, `tier2`, `tier3` | composed (see [Tier composition](#tier-composition)): tier3 when `feature` or `refactor` reaches its intent threshold; otherwise the tier scorer's pick; final tier = max(that, structural floor); none → abstain | 1 |
|
|
44
44
|
| Workflow | the router's strategies (`direct-single`, `iterative-single`, `fable-staged`, `fable-parallel`, `fable-multi-agent-workspace`) | pick one, same rule; an explicit workflow request always wins | 1 |
|
|
45
|
-
|
|
|
45
|
+
| Skill relevance | canonical skills, scored against their descriptions | candidate signals for requirement composition; never a document path/load decision | 3 |
|
|
46
46
|
|
|
47
|
-
The cap of 3 for set stages
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
47
|
+
The cap of 3 for set stages bounds scorer output noise in this proposed model;
|
|
48
|
+
it is not a cap on the number of required workflow bindings. Under issue #297,
|
|
49
|
+
any candidate knowledge signal must still be composed onto a specific ordered
|
|
50
|
+
requirement step before a skill/reference path is disclosed or loaded.
|
|
51
51
|
|
|
52
52
|
Stages run in table order: validity, intent, tier, workflow, skills. Tier
|
|
53
53
|
reads the intent scores, so intent runs before it.
|
package/docs/workflow-runtime.md
CHANGED
|
@@ -6,7 +6,7 @@ Harness uses workflow state to **observe, explain, and remind about semantic con
|
|
|
6
6
|
|
|
7
7
|
The runtime follows **minimal rails, maximum freedom**:
|
|
8
8
|
|
|
9
|
-
- routing **MUST** establish the selected/deferred topology and
|
|
9
|
+
- routing **MUST** establish the selected/deferred topology and normalized knowledge signals for ordered requirement composition;
|
|
10
10
|
- workflow state records useful evidence and unresolved obligations without becoming a cognitive lock;
|
|
11
11
|
- selected-topology required obligations and applicable skill core contracts are semantic **MUSTs**;
|
|
12
12
|
- the agent **MUST** verify before claiming completion and use the single user-visible Harness Status at required phase boundaries for non-trivial work;
|
|
@@ -16,7 +16,7 @@ The runtime follows **minimal rails, maximum freedom**:
|
|
|
16
16
|
|
|
17
17
|
Those cognitive workflow conditions **MUST NOT** trap a session or require a reset command.
|
|
18
18
|
|
|
19
|
-
A semantic MUST is not the same thing as a hard runtime lock. User-visible status,
|
|
19
|
+
A semantic MUST is not the same thing as a hard runtime lock. User-visible status, active-binding obligations, and verification-before-claim can remain mandatory agent contracts while host hooks observe or remind on a best-effort basis. Do not downgrade a semantic obligation to optional advice merely because the host cannot block it.
|
|
20
20
|
|
|
21
21
|
The unified status shape uses a visible Markdown hierarchy:
|
|
22
22
|
|
|
@@ -87,8 +87,12 @@ A workflow may retain `blocked` as a descriptive status recorded by the controll
|
|
|
87
87
|
|
|
88
88
|
## Fable and worktrees
|
|
89
89
|
|
|
90
|
+
Any present `requiredBindings` or `optionalBindings` declaration MUST be an array. Omitted declarations and empty arrays are supported; `null`, objects, strings, booleans, and numbers are rejected before a requirement plan changes workflow state, including fragments without `stepType`.
|
|
91
|
+
|
|
90
92
|
When a Fable topology is selected, its stage graphs, dependencies, write sets, objective checks, synthesis barrier, and required verifier are semantic **MUSTs**. `maxWorkers` remains a **MAY** planning hint and no longer chunks ready stages as a hard concurrency cap.
|
|
91
93
|
|
|
94
|
+
Requirement steps and Fable stages may declare required/optional skill or reference bindings. Runtime output exposes only the active single-agent step or dependency-ready Fable stages. A successful Fable check with unresolved bindings records `binding-unresolved`; after dispositions resolve, the exact check must be rerun before the stage passes. Unknown/unavailable bindings remain visible, while workflow hooks and Stop remain fail-open reminders. A binding disposition is auditable agent evidence, not proof that a host actually loaded or followed the referenced document.
|
|
95
|
+
|
|
92
96
|
For Tier-3/Fable broad mutation, isolation disposition is a semantic **MUST**: use/reuse a verified linked worktree, or record an explicit degraded fallback when isolation is unavailable or the user explicitly chooses to stay in place. Missing isolation is surfaced prominently, but Harness-owned cognitive workflow hooks do not turn it into a self-deadlocking runtime state.
|
|
93
97
|
|
|
94
98
|
## Legacy state
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Workflow: Harness Everything
|
|
2
2
|
|
|
3
|
-
> Route software/project work through the Harness kernel: classify Tier 1/2/3, establish mandatory invariants, then let the agent choose tactics
|
|
3
|
+
> Route software/project work through the Harness kernel: classify Tier 1/2/3, establish mandatory invariants, compose step-scoped knowledge bindings, then let the agent choose tactics. Not general Q&A or non-software writing.
|
|
4
4
|
|
|
5
5
|
Source of truth: `harness-everything/SKILL.md`. Deep dive: `harness-everything/references/triage-and-tiers.md`.
|
|
6
6
|
|
|
@@ -8,7 +8,7 @@ Source of truth: `harness-everything/SKILL.md`. Deep dive: `harness-everything/r
|
|
|
8
8
|
|
|
9
9
|
## 1. Skill Behavior Workflow
|
|
10
10
|
|
|
11
|
-
The kernel establishes the smallest
|
|
11
|
+
The kernel establishes the smallest sufficient workflow before mutation. Normalized keyword signals help compose ordered requirements but never select document paths. Each step declares its required/optional knowledge; the selected topology is mandatory to resolve.
|
|
12
12
|
|
|
13
13
|
```mermaid
|
|
14
14
|
graph TD
|
|
@@ -22,8 +22,8 @@ graph TD
|
|
|
22
22
|
Tier1Route --> Checkpoint["Surface Harness Routing Checkpoint"]
|
|
23
23
|
Tier2Route --> Checkpoint
|
|
24
24
|
Tier3Route --> Checkpoint
|
|
25
|
-
Checkpoint -->
|
|
26
|
-
|
|
25
|
+
Checkpoint --> Decompose["Compose ordered requirement steps and bindings"]
|
|
26
|
+
Decompose --> Execute["Load/resolve only active-step bindings; execute selected topology"]
|
|
27
27
|
Execute --> VerifyGate{Objective evidence supports completion?}
|
|
28
28
|
VerifyGate -->|Yes| Done([Evidence-backed completion])
|
|
29
29
|
VerifyGate -->|No| Diagnose["Diagnose / bounded re-plan"]
|
|
@@ -37,18 +37,18 @@ graph TD
|
|
|
37
37
|
|
|
38
38
|
## 2. Triggering and Routing Path
|
|
39
39
|
|
|
40
|
-
This diagram shows how the `harness-everything` skill is triggered and how it
|
|
40
|
+
This diagram shows how the `harness-everything` skill is triggered and how it composes step-scoped knowledge bindings. Keyword signals are not document selectors; the selected topology is not advisory.
|
|
41
41
|
|
|
42
42
|
```mermaid
|
|
43
43
|
graph LR
|
|
44
44
|
UserPrompt([User Input]) --> Router["harness-everything / kernel-router.js + tier-router.js"]
|
|
45
45
|
|
|
46
|
-
Router -->|
|
|
47
|
-
Router -->|
|
|
48
|
-
Router -->|Large / Multi-source / Multi-session| Ch3["
|
|
49
|
-
Router -->|Architecture / Plan review| Ch4["
|
|
46
|
+
Router -->|Behavior change| Ch1["Bind TDD to the behavior-change step"]
|
|
47
|
+
Router -->|Verification| Ch2["Bind verification-loop to the verification step"]
|
|
48
|
+
Router -->|Large / Multi-source / Multi-session| Ch3["Fable stages expose bindings only when dependencies pass"]
|
|
49
|
+
Router -->|Architecture / Plan review| Ch4["Bind domain skills/references only to their consuming step"]
|
|
50
50
|
|
|
51
|
-
Ch1 --> Invariants["Invariants: route-before-execution, verify-before-claim, re-plan-after-x3,
|
|
51
|
+
Ch1 --> Invariants["Invariants: route-before-execution, verify-before-claim, re-plan-after-x3, resolve-active-required-bindings, resolve-selected-workflow"]
|
|
52
52
|
Ch2 --> Invariants
|
|
53
53
|
Ch3 --> Invariants
|
|
54
54
|
Ch4 --> Invariants
|
|
@@ -65,9 +65,9 @@ graph LR
|
|
|
65
65
|
graph TD
|
|
66
66
|
Input1["User: 'Add a new settings page'"] --> Router1["kernel-router.js executes"]
|
|
67
67
|
Router1 --> Triage2["Triaged as Tier 2: Standard Task"]
|
|
68
|
-
Triage2 --> Checkpoint1["
|
|
69
|
-
Checkpoint1 --> Eval1["
|
|
70
|
-
Eval1 --> Exec1["
|
|
68
|
+
Triage2 --> Checkpoint1["Planning signals help decompose ordered requirements"]
|
|
69
|
+
Checkpoint1 --> Eval1["Bind TDD to behavior change; verification-loop to verification"]
|
|
70
|
+
Eval1 --> Exec1["Load current-step binding, execute, and verify before claiming done"]
|
|
71
71
|
Exec1 --> Done1([Feature delivered with objective evidence])
|
|
72
72
|
|
|
73
73
|
Input2["User: 'Refactor DB architecture across services'"] --> Router2["kernel-router.js executes"]
|
|
@@ -91,7 +91,7 @@ Escape applies only to genuinely uncovered workflow scope with explicit scope +
|
|
|
91
91
|
To ensure that the `harness-everything` skill is operating in strict compliance with Harness OS design laws, verify the following:
|
|
92
92
|
|
|
93
93
|
- [ ] **Route before execution**: scope/tier and smallest applicable workflow established before mutation.
|
|
94
|
-
- [ ] **
|
|
94
|
+
- [ ] **Bind knowledge by step**: required/optional skills and references are declared per ordered requirement; only active paths are loaded/disclosed.
|
|
95
95
|
- [ ] **Resolve selected workflow**: the selected topology executed to resolution; escape only for uncovered scope with evidence.
|
|
96
96
|
- [ ] **Verify before claim**: objective evidence (tests, checks, quoted tool output) supports completion.
|
|
97
97
|
- [ ] **Re-plan on repetition**: after 3 same-signature failures, `zoom-out` reflection completed before resuming.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-trivial-tier1-001
|
|
2
2
|
name: Trivial edit routes to Tier 1
|
|
3
|
-
description: "Route software/project work through Harness: classify tier/topology,
|
|
3
|
+
description: "Route software/project work through Harness: classify tier/topology, compose active-step knowledge bindings, apply semantic obligations, and keep tactics flexible. Use for software triage/routing/re-routing; not general Q&A or non-software writing."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
- tier-1
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-standard-tier2-001
|
|
2
2
|
name: Standard bug fix routes to Tier 2
|
|
3
|
-
description: "Route software/project work through Harness: classify tier/topology,
|
|
3
|
+
description: "Route software/project work through Harness: classify tier/topology, compose active-step knowledge bindings, apply semantic obligations, and keep tactics flexible. Use for software triage/routing/re-routing; not general Q&A or non-software writing."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
- tier-2
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
id: positive-macro-tier3-001
|
|
2
2
|
name: Architectural task routes to Tier 3
|
|
3
|
-
description: "Route software/project work through Harness: classify tier/topology,
|
|
3
|
+
description: "Route software/project work through Harness: classify tier/topology, compose active-step knowledge bindings, apply semantic obligations, and keep tactics flexible. Use for software triage/routing/re-routing; not general Q&A or non-software writing."
|
|
4
4
|
tags:
|
|
5
5
|
- positive-trigger
|
|
6
6
|
- tier-3
|
|
@@ -37,6 +37,8 @@ Every planned stage declares dependency and write scope before dispatch:
|
|
|
37
37
|
"outputPath": "path/to/artifact-or-null",
|
|
38
38
|
"dependsOn": ["stage-1"],
|
|
39
39
|
"writeSet": ["src/auth", "tests/auth.test.js"],
|
|
40
|
+
"requiredBindings": [{ "id": "tdd", "path": "tdd/SKILL.md" }],
|
|
41
|
+
"optionalBindings": [{ "id": "security-review", "path": "security-review/SKILL.md" }],
|
|
40
42
|
"checkCommand": "node --test tests/auth.test.js",
|
|
41
43
|
"passCondition": "exit 0",
|
|
42
44
|
"failureReturn": "failure summary + evidence + missing prerequisite"
|
|
@@ -48,6 +50,7 @@ Rules:
|
|
|
48
50
|
- `stageId`, `goal`, `agent`, and `task` are required.
|
|
49
51
|
- `dependsOn` is always explicit. Use `[]` for a root stage.
|
|
50
52
|
- `writeSet` is always explicit. Use `[]` for a read-only stage.
|
|
53
|
+
- `requiredBindings` and `optionalBindings` are optional arrays. Each entry is a stable `id` and may declare a concrete repository-relative `path`; paths reject absolute names, globs, and traversal. Availability is `available` only when the path resolves to a readable file: workspace references resolve only under the workspace, while recognized Harness bindings (a registered id at its registered path, or a skill id naming its own `<id>/SKILL.md`) also search installed Harness skill roots (`missing` otherwise; `unverified` when no hook runtime is installed). An id without a path resolves through self-evolved manifest metadata, else remains visible with unknown availability. A binding that does not resolve cannot be recorded as `loaded`.
|
|
51
54
|
- `writeSet` contains concrete repository-relative path scopes, not globs,
|
|
52
55
|
absolute paths, or `..` traversal.
|
|
53
56
|
- a directory scope covers descendants; overlapping scopes cannot run in the
|
|
@@ -60,7 +63,7 @@ Rules:
|
|
|
60
63
|
|
|
61
64
|
## Workflow-plan consumer
|
|
62
65
|
|
|
63
|
-
For a session with an active mandatory workflow, write the stage array to the router-displayed `workflow-stages.json` path and invoke its `workflow-disposition.js start --session-id <id>` command. This binds the run to the active `workflowId` and bound workspace even after entering a linked worktree. Every required stage needs an objective check, and an independent plan needs a read-only `fable-verifier` stage depending on all other stages. Stage results need observed worker identities and numeric exit codes; missing metadata is
|
|
66
|
+
For a session with an active mandatory workflow, write the stage array to the router-displayed `workflow-stages.json` path and invoke its `workflow-disposition.js start --session-id <id>` command. This binds the run to the active `workflowId` and bound workspace even after entering a linked worktree. The start/check output exposes bindings only for dependency-ready stages; future-stage paths stay undisclosed. Resolve an active binding with `workflow-disposition.js stage-binding --stage-id <id> --binding-id <id> --disposition loaded|not-needed|unavailable --evidence <evidence>`. A successful check with unresolved bindings records `binding-unresolved`; resolve them, then rerun the exact check before the stage can pass. This state is a reminder, not a tool or Stop lock. Every required stage needs an objective check, and an independent plan needs a read-only `fable-verifier` stage depending on all other stages. Stage results need observed worker identities and numeric exit codes; missing metadata is unresolved, not success.
|
|
64
67
|
|
|
65
68
|
For standalone use without an active session contract, write the router contract and stage array to files and run:
|
|
66
69
|
|
package/fable-mode/SKILL.md
CHANGED
|
@@ -4,7 +4,7 @@ description: "Stage large tasks through plans, named fable agents, failable chec
|
|
|
4
4
|
license: Apache-2.0
|
|
5
5
|
metadata:
|
|
6
6
|
author: Miya Daniel
|
|
7
|
-
version: 0.
|
|
7
|
+
version: 0.31.0
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# Fable Mode
|
|
@@ -29,17 +29,15 @@ metadata:
|
|
|
29
29
|
|
|
30
30
|
## Select Operation First
|
|
31
31
|
|
|
32
|
-
For explanation/profile-only lookup, read only `<this-skill-dir>/references/profile-lookup.md` and answer. No stage state, delegation, or selector source reads. Quoted aliases do not start Fable. Mixed execution retains its topology.
|
|
33
|
-
|
|
34
|
-
Execution follows the workflow below.
|
|
32
|
+
For explanation/profile-only lookup, read only `<this-skill-dir>/references/profile-lookup.md` and answer. No stage state, delegation, or selector source reads. Quoted aliases do not start Fable. Mixed execution retains its topology; execution follows the workflow below.
|
|
35
33
|
|
|
36
34
|
## Workflow
|
|
37
35
|
|
|
38
36
|
1. Discover the runtime and lock the authorized file scope before edits.
|
|
39
|
-
2.
|
|
37
|
+
2. Per stage, set artifact, check, and required/optional bindings; dependency-ready stages disclose them, passing once resolved. Gaps never lock tools or Stop; advisory replans never halt. Rule-of-3 covers same-signature failures.
|
|
40
38
|
3. Resolve the requested behavior profile with `model-selector.js`; `opus`, `sonnet`/`sonnect`, and `haiku` are compatibility aliases, not model bindings.
|
|
41
|
-
4. Delegate by profile: orchestrator coordinates, reasoning
|
|
42
|
-
5.
|
|
43
|
-
6. Run
|
|
39
|
+
4. Delegate by profile: orchestrator coordinates, reasoning takes bounded judgment, mechanical takes low-ambiguity work; workers never spawn workers.
|
|
40
|
+
5. The host runtime-model floor is advisory; record it, never block on model choice alone.
|
|
41
|
+
6. Run stage checks, cold-review high-stakes artifacts, and audit profile/runtime/fallback fields.
|
|
44
42
|
|
|
45
43
|
Deep dive: <this-skill-dir>/references/model-matrix.md
|