harness-everything 0.21.0 → 0.22.1

package/.claude-plugin/marketplace.json

@@ -10,7 +10,7 @@
       "name": "harness-everything",
       "source": "./",
       "description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
-      "version": "0.21.0",
+      "version": "0.22.1",
       "author": {
         "name": "Miya Daniel"
       },

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "harness-everything",
   "description": "Harness OS: an orchestrated agent operating system — tier routing, circuit breaker guardrails, verification gates, WAL session memory, and self-evolving workspace rules.",
-  "version": "0.21.0",
+  "version": "0.22.1",
   "author": {
     "name": "Miya Daniel"
   },

package/CHANGELOG.md

@@ -2,6 +2,24 @@
 
 All notable changes to this project are documented in this file.
 
+## [0.22.1](https://github.com/dyphn1/Harness-everything/compare/v0.22.0...v0.22.1) (2026-09-23)
+
+### Performance Improvements
+
+* **router:** pre-score system one asynchronously in the hook entry ([fcf2de2](https://github.com/dyphn1/Harness-everything/commit/fcf2de2a843d106636c10a318d0832c2b57d64f6)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
+
+## [0.22.0](https://github.com/dyphn1/Harness-everything/compare/v0.21.0...v0.22.0) (2026-09-23)
+
+### Features
+
+* **router:** add resident system one provider ([eee8cb7](https://github.com/dyphn1/Harness-everything/commit/eee8cb763d2ed65722a2578c83700b02bdc3e668)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
+
+### Bug Fixes
+
+* **distribution:** keep python bytecode out of every package ([58135ae](https://github.com/dyphn1/Harness-everything/commit/58135aef92089d731eb44d55efb0ae77ea4d64dd)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
+* **router:** harden resident system one server from adversarial review ([ee5ab0f](https://github.com/dyphn1/Harness-everything/commit/ee5ab0fec9f03808f12a88e66b769b210803baa2)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
+* **router:** replace the whole windows dacl for resident state ([e59af73](https://github.com/dyphn1/Harness-everything/commit/e59af735de1337b3bcb10cbca05500ab6b03b64c)), closes [#233](https://github.com/dyphn1/Harness-everything/issues/233)
+
 ## [0.21.0](https://github.com/dyphn1/Harness-everything/compare/v0.20.7...v0.21.0) (2026-09-23)
 
 ### Features

package/README.md

@@ -270,6 +270,7 @@ For a deep dive into individual modules and the underlying philosophy, explore o
 * [Harness Routing & Triage](docs/routing.md): Detailed trigger criteria for Tiers 1, 2, and 3.
 * [Harness Reflection & Memory](docs/reflection.md): WAL session handoffs and workspace rules immunization.
 * [Harness Audit Log](docs/audit.md): Dated historical self-audit scorecards, methodology, and per-cycle change log.
+* [System One Routing (experimental, off by default)](docs/system-one-routing.md): Optional local option scorer for tier routing, its gates and evidence.
 
 Fable model selection is documented in [fable-mode/references/model-matrix.md](fable-mode/references/model-matrix.md); the explicit entrypoints are `fable-haiku`, `fable-sonnet`, and `fable-opus`.
 
@@ -277,6 +278,35 @@ Maintainers **MUST** follow [RELEASING.md](RELEASING.md) for tag-driven npm rele
 
 ---
 
+## Experimental: System One routing (off by default — keep it off for now)
+
+Issue [#233](https://github.com/dyphn1/Harness-everything/issues/233) adds an optional local "System One" scorer. It scores the fixed tier catalog for each prompt. The lexical router stays authoritative:
+- `HARNESS_SYSTEM_ONE_MODE` defaults to `off`.
+- `shadow` only logs a diagnostic.
+- `prefer` can raise a tier but never lower it below the deterministic structural floor, and only for a `harness-routing-v1` checkpoint.
+
+**Honest status: it is not useful yet, so do not enable it for normal work.**
+
+- **No usable model yet.** The only available checkpoint, `cua-ai/cua-s1-forms`, is a form-filling model, so every decision abstains as `domain-mismatch` and routing never changes. On historical routing keywords its top-1 skill choice is at chance level: 8.2% against 11.6% for a random guess, and 5.9% against 3.8% on eval prompts. It is kept only to prove the pipeline. The feature should stay `off` until a Harness-trained checkpoint passes the Phase 4 gates.
+- **It costs something when on.** The resident transport keeps one Python/PyTorch server per manifest alive until 30 minutes idle. The measured cost on one Windows CPU host is about 85–110 ms per prompt, and the warm p95 of 108.6 ms misses the 100 ms target. The first prompt after a cold start uses the lexical route while the server loads.
+- **Setup downloads a lot.** `npm run system-one:install` needs Python 3.11–3.13 and downloads CPU PyTorch (~124 MB on Windows), the pinned `cua_s1` source and the pinned checkpoint. It never runs from hooks and never turns the mode on.
+
+**Enabling it depends on the host, and most options are process-wide:**
+
+| Surface | How System One gets its configuration | Caveat |
+| --- | --- | --- |
+| Claude Code (plugin or `--claude` install) | `env` in `~/.claude/settings.json` (`HARNESS_SYSTEM_ONE_MODE`, `HARNESS_SYSTEM_ONE_CONFIG`) | The values reach hooks **and every command the agent runs** (tests, builds), and are hot-reloaded into running sessions. |
+| Codex (plugin or `--codex` install) | Hooks inherit the environment of the `codex` process | A Codex started from Claude Code inherits Claude's `env`. A Codex started from an ordinary terminal only sees the variables if they are set **at user/OS level** (Windows user environment variables, or your shell profile on macOS/Linux), which then applies to every program you start. |
+| Skills-only / public OpenAI bundle | No hooks | It runs only if the agent itself runs `harness-everything/scripts/kernel-router.js`. |
+
+**Evidence so far:**
+- **Mechanism:** Linux, macOS and Windows CI.
+- **Real CPU inference:** on one Windows machine.
+- **Live host:** one Windows + Claude Code observation, where a hook-spawned server outlived the host. macOS/Linux hosts and Codex hosts are not live-verified.
+- **Details:** [docs/system-one-routing.md](docs/system-one-routing.md) and [benchmarks/results/system-one/](benchmarks/results/system-one/pre-merge-review-2026-09-23/README.md).
+
+---
+
 ## Benchmarks & Testing
 
 **If you are an agent asked to verify a Harness install, start at [VERIFICATION.md](VERIFICATION.md), not here.** It separates package/integrity checks, mechanism evidence, live-host evidence, and behavioral evidence so one kind of pass is not mistaken for another.

package/benchmarks/results/system-one/cua-s1-forms-resident-async-cpu-2026-09-23/README.md

@@ -0,0 +1,87 @@
+# System One: async hook pre-score on the resident CPU provider (2026-09-23)
+
+Evidence layer: **local real-checkpoint inference through the resident transport**,
+paired against the previous synchronous client. It is not live-host evidence, not
+a reviewed holdout, and not a Harness routing quality claim. Issue
+[#233](https://github.com/dyphn1/Harness-everything/issues/233); previous resident
+run: [cua-s1-forms-resident-cpu-2026-09-23](../cua-s1-forms-resident-cpu-2026-09-23/README.md).
+
+## Change under test
+
+The hook entry (`tier-router.js` run as a script) now pre-scores over a direct
+async loopback call (`scoreAsync`) and routes with that result. The synchronous
+client starts a worker thread for every call. Decisions, reason mapping and the
+router contract are unchanged; see [docs/system-one-routing.md](../../../../docs/system-one-routing.md).
+
+## Setup
+
+Same host, venv, pinned `cua-s1` source (`pinned`) and `cua-ai/cua-s1-forms`
+checkpoint as the previous runs: Intel Core Ultra 5 225H, Windows 11 (win32
+10.0.26200 x64), Node v24.18.0, CPython 3.13.6, one torch thread. "main" is
+`f9d70ed`, which uses the synchronous worker bridge.
+
+## Evaluator
+
+```bash
+node scripts/evaluate-system-one.js benchmarks/fixtures/system-one-routing.json \
+  ~/.agents/harness-everything/system-one/manifest.json \
+  benchmarks/results/system-one/cua-s1-forms-resident-async-cpu-2026-09-23/report.json
+```
+
+[`report.json`](report.json) is the unmodified output of this command. The
+server was not running beforehand, so the evaluator started it, waited for
+readiness, and stopped it afterwards. That gives 32 warm samples with p50
+31.5 ms and p95 76.6 ms. `warmLatency` passes, repeatability is 1.0 and
+`sourceProvenance` passes. Decisions are unchanged: 30 are `domain-mismatch`
+and 2 are the over-limit `provider-exit`. Coverage is 0 and `rolloutReady` is false.
+
+**A single run does not separate the two clients.** main, run once right
+after this report under the same conditions, also passed (p95 75.9 ms). The
+earlier 108.6 ms was also a single run. So [`paired-latency.json`](paired-latency.json) records
+three alternating rounds against one already-warm server:
+
+| Round | main p50 / p95 | branch p50 / p95 |
+| --- | --- | --- |
+| 1 | 50.8 / 82.3 ms | 24.6 / 47.6 ms |
+| 2 | 51.4 / 75.9 ms | 26.8 / 51.1 ms |
+| 3 | 58.4 / **116.8** ms | 24.3 / 62.9 ms |
+
+main fails the ≤100 ms gate in one of three rounds and the branch in none. The
+branch roughly halves p50. The branch's single worst sample (189.6 ms, round 3)
+shows that outliers remain. Three rounds of 32 samples are a small sample.
+
+## Hook entry
+
+**Fresh process.** This matches how the hook runs. Each of 20 fresh processes
+loads the modules and scores once.
+
+| | module load p50 | one score p50 / p95 |
+| --- | --- | --- |
+| sync | 9 ms | 77.8 / 105.4 ms |
+| async | 9 ms | 35.9 / 67.6 ms |
+
+**Wall time.** `kernel-router-core.js "fix a typo in README"` runs for 40
+rounds against a warm server. Each round runs `off`, main `shadow` and branch
+`shadow` in turn. Overhead is paired against `off` in the same round.
+
+| | p50 | p95 |
+| --- | --- | --- |
+| `off` | 182 ms | 339 ms |
+| main `shadow` overhead | +63 ms | +179 ms |
+| branch `shadow` overhead | **+44 ms** | **+111 ms** |
+
+The router plans were identical in all 40 rounds for all three variants, and
+every shadow diagnostic was `domain-mismatch`.
+
+## Limitations
+
+- These are single-host Windows numbers with high run-to-run variance.
+  macOS/Linux and other CPUs are not measured.
+- Most of the remaining ~44 ms hook overhead is first-call cost in a fresh
+  process (connect plus a cold client path) on top of server scoring (≈7–12 ms).
+  It is not reduced further here.
+- The seed corpus is 16 unreviewed cases, and the forms checkpoint cannot drive
+  Harness routing. The `reviewedHoldout`, `coverage`, `acceptedPrecision`,
+  `macroF1`, `policyEvidence` and `liveHostEvidence` gates remain unmet.
+- The hook host was not live-tested for this change. Only the CLI entry points
+  above were run.

package/benchmarks/results/system-one/cua-s1-forms-resident-async-cpu-2026-09-23/paired-latency.json

@@ -0,0 +1,107 @@
+{
+  "schemaVersion": 1,
+  "note": "Summaries of unmodified evaluator reports; each round runs main (sync worker bridge, f9d70ed) then this branch (async) against one already-warm resident server.",
+  "evaluatorRounds": [
+    {
+      "main": {
+        "p50Ms": 50.8,
+        "p95Ms": 82.3,
+        "maxMs": 105.8,
+        "warmLatencyGate": true,
+        "agreement": 1
+      },
+      "branch": {
+        "p50Ms": 24.6,
+        "p95Ms": 47.6,
+        "maxMs": 50.8,
+        "warmLatencyGate": true,
+        "agreement": 1
+      }
+    },
+    {
+      "main": {
+        "p50Ms": 51.4,
+        "p95Ms": 75.9,
+        "maxMs": 90.5,
+        "warmLatencyGate": true,
+        "agreement": 1
+      },
+      "branch": {
+        "p50Ms": 26.8,
+        "p95Ms": 51.1,
+        "maxMs": 52.1,
+        "warmLatencyGate": true,
+        "agreement": 1
+      }
+    },
+    {
+      "main": {
+        "p50Ms": 58.4,
+        "p95Ms": 116.8,
+        "maxMs": 155.7,
+        "warmLatencyGate": false,
+        "agreement": 1
+      },
+      "branch": {
+        "p50Ms": 24.3,
+        "p95Ms": 62.9,
+        "maxMs": 189.6,
+        "warmLatencyGate": true,
+        "agreement": 1
+      }
+    }
+  ],
+  "freshProcessScore": {
+    "runs": 20,
+    "sync": {
+      "moduleLoadMs": {
+        "p50": 9,
+        "p95": 23.6
+      },
+      "scoreMs": {
+        "p50": 77.8,
+        "p95": 105.4
+      }
+    },
+    "async": {
+      "moduleLoadMs": {
+        "p50": 9,
+        "p95": 22.7
+      },
+      "scoreMs": {
+        "p50": 35.9,
+        "p95": 67.6
+      }
+    }
+  },
+  "hookWallTime": {
+    "rounds": 40,
+    "prompt": "fix a typo in README",
+    "offMs": {
+      "p50": 182,
+      "p95": 339
+    },
+    "mainShadowMs": {
+      "p50": 236,
+      "p95": 343
+    },
+    "branchShadowMs": {
+      "p50": 224,
+      "p95": 311
+    },
+    "pairedOverheadVsOffMs": {
+      "main": {
+        "p50": 63,
+        "p95": 179
+      },
+      "branch": {
+        "p50": 44,
+        "p95": 111
+      }
+    },
+    "identicalPlans": 40,
+    "shadowReasons": {
+      "domain-mismatch": 80
+    }
+  }
+}