harness-everything 0.22.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +12 -0
- package/README.md +30 -0
- package/benchmarks/fixtures/system-one-holdout-draft.json +2039 -0
- package/benchmarks/fixtures/system-one-holdout-reviews.json +1587 -0
- package/benchmarks/fixtures/system-one-holdout.json +7315 -0
- package/benchmarks/results/system-one/cua-s1-forms-resident-async-cpu-2026-09-23/README.md +87 -0
- package/benchmarks/results/system-one/cua-s1-forms-resident-async-cpu-2026-09-23/paired-latency.json +107 -0
- package/benchmarks/results/system-one/cua-s1-forms-resident-async-cpu-2026-09-23/report.json +856 -0
- package/benchmarks/results/system-one/lexical-baseline-reviewed-holdout-2026-09-23/README.md +47 -0
- package/benchmarks/results/system-one/lexical-baseline-reviewed-holdout-2026-09-23/report.json +5222 -0
- package/benchmarks/results/system-one/pre-merge-review-2026-09-23/README.md +92 -0
- package/benchmarks/results/system-one/pre-merge-review-2026-09-23/haiku-spec-conformance.md +241 -0
- package/benchmarks/results/system-one/pre-merge-review-2026-09-23/luna-adversarial.md +212 -0
- package/benchmarks/results/system-one/pre-merge-review-2026-09-23/skill-top1-probe.js +83 -0
- package/benchmarks/results/system-one/pre-merge-review-2026-09-23/skill-top1-probe.json +7700 -0
- package/ci/fixtures/system-one-worker-probe.js +15 -0
- package/ci/mechanism-system-one-corpus.test.js +154 -0
- package/ci/mechanism-system-one-resident.test.js +114 -3
- package/ci/mechanism-system-one-router.test.js +25 -0
- package/docs/platform-capabilities.md +7 -2
- package/docs/system-one-corpus.md +103 -0
- package/docs/system-one-routing.md +25 -5
- package/evals/pilot-router-nonsoftware/graders/no-tier-language.md +6 -0
- package/evals/pilot-router-nonsoftware/graders/skill-not-fired.md +9 -0
- package/evals/pilot-router-nonsoftware/prompt.md +8 -0
- package/evals/pilot-router-tier3/graders/skill-fired.md +6 -0
- package/evals/pilot-router-tier3/prompt.md +8 -0
- package/evals/pilot-tdd-bugfix/graders/skill-fired.md +6 -0
- package/evals/pilot-tdd-bugfix/prompt.md +8 -0
- package/evals/pilot-tdd-trivia/graders/no-red-phase-claim.md +7 -0
- package/evals/pilot-tdd-trivia/graders/skill-not-fired.md +9 -0
- package/evals/pilot-tdd-trivia/prompt.md +8 -0
- package/evals/pilot-verify-discussion/graders/no-verification-claim.md +7 -0
- package/evals/pilot-verify-discussion/graders/skill-not-fired.md +9 -0
- package/evals/pilot-verify-discussion/prompt.md +8 -0
- package/evals/pilot-verify-preship/graders/skill-fired.md +6 -0
- package/evals/pilot-verify-preship/prompt.md +8 -0
- package/harness-everything/SKILL.md +1 -1
- package/harness-everything/scripts/system-one/provider.js +14 -2
- package/harness-everything/scripts/system-one/resident.js +26 -2
- package/harness-everything/scripts/system-one/router.js +17 -1
- package/harness-everything/scripts/tier-router.js +13 -4
- package/opencode-plugin/plugin.json +1 -1
- package/package.json +1 -1
- package/plugins/harness-everything/.codex-plugin/plugin.json +1 -1
- package/plugins/harness-everything/plugin.json +1 -1
- package/plugins/harness-everything/skills/harness-everything/SKILL.md +1 -1
- package/plugins/harness-everything/skills/harness-everything/scripts/system-one/provider.js +14 -2
- package/plugins/harness-everything/skills/harness-everything/scripts/system-one/resident.js +26 -2
- package/plugins/harness-everything/skills/harness-everything/scripts/system-one/router.js +17 -1
- package/plugins/harness-everything/skills/harness-everything/scripts/tier-router.js +13 -4
- package/sbom.cdx.json +7 -7
- package/scripts/evaluate-system-one.js +6 -6
- package/scripts/system-one-corpus.js +99 -0
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"name": "harness-everything",
|
|
11
11
|
"source": "./",
|
|
12
12
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
13
|
-
"version": "0.
|
|
13
|
+
"version": "0.23.0",
|
|
14
14
|
"author": {
|
|
15
15
|
"name": "Miya Daniel"
|
|
16
16
|
},
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "harness-everything",
|
|
3
3
|
"description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.23.0",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Miya Daniel"
|
|
7
7
|
},
|
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,18 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project are documented in this file.
|
|
4
4
|
|
|
5
|
+
## [0.23.0](https://github.com/dyphn1/Harness-everything/compare/v0.22.1...v0.23.0) (2026-09-23)
|
|
6
|
+
|
|
7
|
+
### Features
|
|
8
|
+
|
|
9
|
+
* **router:** assemble the system one holdout from human review ([c877e9f](https://github.com/dyphn1/Harness-everything/commit/c877e9f5248ea413ce8252f98f672376c83ef534)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
|
|
10
|
+
|
|
11
|
+
## [0.22.1](https://github.com/dyphn1/Harness-everything/compare/v0.22.0...v0.22.1) (2026-09-23)
|
|
12
|
+
|
|
13
|
+
### Performance Improvements
|
|
14
|
+
|
|
15
|
+
* **router:** pre-score system one asynchronously in the hook entry ([fcf2de2](https://github.com/dyphn1/Harness-everything/commit/fcf2de2a843d106636c10a318d0832c2b57d64f6)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
|
|
16
|
+
|
|
5
17
|
## [0.22.0](https://github.com/dyphn1/Harness-everything/compare/v0.21.0...v0.22.0) (2026-09-23)
|
|
6
18
|
|
|
7
19
|
### Features
|
package/README.md
CHANGED
|
@@ -270,6 +270,7 @@ For a deep dive into individual modules and the underlying philosophy, explore o
|
|
|
270
270
|
* [Harness Routing & Triage](docs/routing.md): Detailed trigger criteria for Tiers 1, 2, and 3.
|
|
271
271
|
* [Harness Reflection & Memory](docs/reflection.md): WAL session handoffs and workspace rules immunization.
|
|
272
272
|
* [Harness Audit Log](docs/audit.md): Dated historical self-audit scorecards, methodology, and per-cycle change log.
|
|
273
|
+
* [System One Routing (experimental, off by default)](docs/system-one-routing.md): Optional local option scorer for tier routing, its gates and evidence.
|
|
273
274
|
|
|
274
275
|
Fable model selection is documented in [fable-mode/references/model-matrix.md](fable-mode/references/model-matrix.md); the explicit entrypoints are `fable-haiku`, `fable-sonnet`, and `fable-opus`.
|
|
275
276
|
|
|
@@ -277,6 +278,35 @@ Maintainers **MUST** follow [RELEASING.md](RELEASING.md) for tag-driven npm rele
|
|
|
277
278
|
|
|
278
279
|
---
|
|
279
280
|
|
|
281
|
+
## Experimental: System One routing (off by default — keep it off for now)
|
|
282
|
+
|
|
283
|
+
Issue [#233](https://github.com/dyphn1/Harness-everything/issues/233) adds an optional local "System One" scorer. It scores the fixed tier catalog for each prompt. The lexical router stays authoritative:
|
|
284
|
+
- `HARNESS_SYSTEM_ONE_MODE` defaults to `off`.
|
|
285
|
+
- `shadow` only logs a diagnostic.
|
|
286
|
+
- `prefer` can raise a tier but never lower it below the deterministic structural floor, and only for a `harness-routing-v1` checkpoint.
|
|
287
|
+
|
|
288
|
+
**Honest status: it is not useful yet, so do not enable it for normal work.**
|
|
289
|
+
|
|
290
|
+
- **No usable model yet.** The only available checkpoint, `cua-ai/cua-s1-forms`, is a form-filling model, so every decision abstains as `domain-mismatch` and routing never changes. On historical routing keywords its top-1 skill choice is at chance level: 8.2% against 11.6% for a random guess, and 5.9% against 3.8% on eval prompts. It is kept only to prove the pipeline. The feature should stay `off` until a Harness-trained checkpoint passes the Phase 4 gates.
|
|
291
|
+
- **It costs something when on.** The resident transport keeps one Python/PyTorch server per manifest alive until 30 minutes idle. The measured cost on one Windows CPU host is about 85–110 ms per prompt, and the warm p95 of 108.6 ms misses the 100 ms target. The first prompt after a cold start uses the lexical route while the server loads.
|
|
292
|
+
- **Setup downloads a lot.** `npm run system-one:install` needs Python 3.11–3.13 and downloads CPU PyTorch (~124 MB on Windows), the pinned `cua_s1` source and the pinned checkpoint. It never runs from hooks and never turns the mode on.
|
|
293
|
+
|
|
294
|
+
**Enabling it depends on the host, and most options are process-wide:**
|
|
295
|
+
|
|
296
|
+
| Surface | How System One gets its configuration | Caveat |
|
|
297
|
+
| --- | --- | --- |
|
|
298
|
+
| Claude Code (plugin or `--claude` install) | `env` in `~/.claude/settings.json` (`HARNESS_SYSTEM_ONE_MODE`, `HARNESS_SYSTEM_ONE_CONFIG`) | The values reach hooks **and every command the agent runs** (tests, builds), and are hot-reloaded into running sessions. |
|
|
299
|
+
| Codex (plugin or `--codex` install) | Hooks inherit the environment of the `codex` process | A Codex started from Claude Code inherits Claude's `env`. A Codex started from an ordinary terminal only sees the variables if they are set **at user/OS level** (Windows user environment variables, or your shell profile on macOS/Linux), which then applies to every program you start. |
|
|
300
|
+
| Skills-only / public OpenAI bundle | No hooks | It runs only if the agent itself runs `harness-everything/scripts/kernel-router.js`. |
|
|
301
|
+
|
|
302
|
+
**Evidence so far:**
|
|
303
|
+
- **Mechanism:** Linux, macOS and Windows CI.
|
|
304
|
+
- **Real CPU inference:** on one Windows machine.
|
|
305
|
+
- **Live host:** one Windows + Claude Code observation, where a hook-spawned server outlived the host. macOS/Linux hosts and Codex hosts are not live-verified.
|
|
306
|
+
- **Details:** [docs/system-one-routing.md](docs/system-one-routing.md) and [benchmarks/results/system-one/](benchmarks/results/system-one/pre-merge-review-2026-09-23/README.md).
|
|
307
|
+
|
|
308
|
+
---
|
|
309
|
+
|
|
280
310
|
## Benchmarks & Testing
|
|
281
311
|
|
|
282
312
|
**If you are an agent asked to verify a Harness install, start at [VERIFICATION.md](VERIFICATION.md), not here.** It separates package/integrity checks, mechanism evidence, live-host evidence, and behavioral evidence so one kind of pass is not mistaken for another.
|