token-harness 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +117 -16
- package/package.json +1 -1
- package/sbom.json +2 -2
- package/token-harness.mjs +1873 -277
package/README.md
CHANGED
|
@@ -32,6 +32,8 @@ neither installed nor configured by Token Harness.
|
|
|
32
32
|
| [LiteLLM](https://github.com/BerriAI/litellm) | Model routing, fallbacks, budgets, and usage telemetry | Not active — candidate |
|
|
33
33
|
| [RouteLLM](https://github.com/lm-sys/RouteLLM) | Route simpler requests to less expensive models | Not active — candidate |
|
|
34
34
|
| [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) | Route by task, complexity, tools, and deployment locality | Not active — candidate |
|
|
35
|
+
| [Claude Code Router](https://github.com/musistudio/claude-code-router) | Route coding-agent requests across models and providers with effort-based rules and fallback chains | Not active — candidate · high priority |
|
|
36
|
+
| [LLMRouter](https://github.com/ulab-uiuc/LLMRouter) | Select the model by task complexity, cost, and quality across routing strategies | Not active — candidate · high priority |
|
|
35
37
|
| [Headroom](https://github.com/headroomlabs-ai/headroom) | Compress tool, MCP, file, and RAG payloads | Not active — candidate |
|
|
36
38
|
| [Context Mode](https://github.com/mksglu/context-mode) | Keep raw tool results outside model context | Not active — candidate |
|
|
37
39
|
| [LLMLingua](https://github.com/microsoft/LLMLingua) | Compress long prompts and context | Not active — candidate |
|
|
@@ -40,7 +42,9 @@ neither installed nor configured by Token Harness.
|
|
|
40
42
|
Candidate status means only that the project has identified a useful optimization layer. A tool
|
|
41
43
|
becomes active only after its installation, conflicts, rollback behavior, verification, and
|
|
42
44
|
metrics attribution have been implemented and tested. Token Harness never installs a candidate
|
|
43
|
-
merely because it is present on the machine.
|
|
45
|
+
merely because it is present on the machine. Rows marked `high priority` are the next intended
|
|
46
|
+
intake; the admission gates each one carries are recorded in
|
|
47
|
+
[docs/provider-landscape.md](docs/provider-landscape.md).
|
|
44
48
|
|
|
45
49
|
## Quick start
|
|
46
50
|
|
|
@@ -83,27 +87,85 @@ npx token-harness doctor
|
|
|
83
87
|
`npx` may download Token Harness into npm's cache, but it does not install or configure RTK,
|
|
84
88
|
HarnessTrim, or a coding agent.
|
|
85
89
|
|
|
90
|
+
### Managed compatibility rows
|
|
91
|
+
|
|
92
|
+
Token Harness changes a harness configuration only when a reviewed compatibility row covers the
|
|
93
|
+
exact provider version, harness version, platform, and configuration schema. Three rows ship, and each
|
|
94
|
+
names the recording it stands on:
|
|
95
|
+
|
|
96
|
+
| Provider | Harness | Platform | Tested versions | Tier |
|
|
97
|
+
| --- | --- | --- | --- | --- |
|
|
98
|
+
| RTK | Claude Code | Windows | rtk 0.44.0, Claude Code 2.1.220 | `canary` |
|
|
99
|
+
| HarnessTrim | Claude Code | Windows | harnesstrim 0.1.0, Claude Code 2.1.220 | `config-only` |
|
|
100
|
+
| HarnessTrim | Codex | Windows | harnesstrim 0.1.0, Codex 0.146.0 | `config-only` |
|
|
101
|
+
|
|
102
|
+
Everything else is refused, and that is the design rather than a gap: `doctor` detects and reports on
|
|
103
|
+
every supported platform, and only the *mutation* is narrower. An uncovered combination exits 9 and
|
|
104
|
+
the diagnostic names what is missing — the reviewed fixture, or the nearest row it does have.
|
|
105
|
+
|
|
106
|
+
What is not covered today, and why:
|
|
107
|
+
|
|
108
|
+
- **macOS and Linux.** No row on either. The recordings a row needs are states of a real machine, and
|
|
109
|
+
a fixture cannot be written from a machine nobody ran. On those platforms `plan` and `apply` refuse;
|
|
110
|
+
install the provider with its own installer and Token Harness will detect, verify, and measure it.
|
|
111
|
+
- **OpenCode, and permanently rather than pending.** Both providers are detected, adopted, verified
|
|
112
|
+
and measured there, and neither is written. RTK reaches OpenCode through a plugin module its own
|
|
113
|
+
installer places globally, which this build has no action for. HarnessTrim's OpenCode installer
|
|
114
|
+
writes a plugin wrapper *and runs an npm install*, so a containment boundary covering what it wrote
|
|
115
|
+
would hold a `node_modules` tree — and that is not a decision deferred for want of a fixture. A
|
|
116
|
+
dependency tree is not configuration, so it cannot be a reviewed write set; snapshotting it on
|
|
117
|
+
every apply to keep the rollback honest would be slow and would be restoring upstream's install
|
|
118
|
+
rather than our change; and excluding it would leave a transaction claiming a reversibility it does
|
|
119
|
+
not have. So the assignment is not producible, and RFC 0003 is explicit about what that means: a
|
|
120
|
+
capability the provider has but cannot be asked for is not an assignable capability. OpenCode stays
|
|
121
|
+
adoption-only by decision.
|
|
122
|
+
- **RTK on Codex.** Not managed, and no row: RTK writes a Claude-shaped hook list and nothing else.
|
|
123
|
+
- **A newer Claude Code.** The range is a single observed version. `2.1.221` reads `unknown-newer` and
|
|
124
|
+
refuses rather than assuming it behaves like `2.1.220`.
|
|
125
|
+
|
|
126
|
+
The recordings are under `tests/fixtures/rows/`, one directory per row, each with a README stating
|
|
127
|
+
which stages exist and which do not.
|
|
128
|
+
|
|
86
129
|
## How the components fit together
|
|
87
130
|
|
|
88
131
|
There are three separate layers. Installing one does not automatically provide the others.
|
|
89
132
|
|
|
90
133
|
| Layer | Examples | Who installs it? |
|
|
91
134
|
| --- | --- | --- |
|
|
92
|
-
| Coding agent (harness) | Claude Code, Codex, OpenCode | You, using the agent's official installer |
|
|
135
|
+
| Coding agent (harness) | Claude Code, Codex, OpenCode, Hermes, Pi | You, using the agent's official installer |
|
|
93
136
|
| Token Harness | `token-harness` | You, from npm or this repository |
|
|
94
|
-
| Optimization provider | RTK, HarnessTrim |
|
|
137
|
+
| Optimization provider | RTK, HarnessTrim | Both can be installed by Token Harness where a compatibility row covers the combination; otherwise install them with their own installers and Token Harness detects and measures them |
|
|
95
138
|
|
|
96
|
-
Token Harness does not install Claude Code, Codex, or
|
|
139
|
+
Token Harness does not install Claude Code, Codex, OpenCode, Hermes, or Pi. Install and run at least one of
|
|
97
140
|
them first so that `token-harness doctor` can detect it.
|
|
98
141
|
|
|
99
|
-
| Provider | Claude Code | Codex | OpenCode | Installed by Token Harness |
|
|
100
|
-
| --- | --- | --- | --- | --- |
|
|
101
|
-
| RTK | Configure, verify, and measure | Not managed | Not managed | **Yes**, for the supported Claude Code path |
|
|
102
|
-
| HarnessTrim | Claude skills only; no reducer hook or reduce-pipe instruction | Detect, adopt, verify, and measure | Detect, adopt, verify, and measure | **Yes**,
|
|
142
|
+
| Provider | Claude Code | Codex | OpenCode | Hermes | Pi | Installed by Token Harness |
|
|
143
|
+
| --- | --- | --- | --- | --- | --- | --- |
|
|
144
|
+
| RTK | Configure, verify, and measure | Not managed | Detect, adopt, verify, and measure | Not managed | Not managed | **Yes**, for the supported Claude Code path |
|
|
145
|
+
| HarnessTrim | Claude skills only; no reducer hook or reduce-pipe instruction | Detect, adopt, verify, and measure | Detect, adopt, verify, and measure | Detect, verify, and measure | Detect, verify, and measure | **Yes**, on a covered row — see above |
|
|
103
146
|
|
|
104
147
|
"Not managed" does not mean the upstream tool cannot support that agent. It means this release
|
|
105
148
|
does not claim ownership of that integration and will not modify it.
|
|
106
149
|
|
|
150
|
+
Hermes is read-only in both directions: the adapter finds the HarnessTrim plugin, reads whether it
|
|
151
|
+
is enabled, and imports the telemetry it writes to `~/.hermes/harnesstrim-metrics.jsonl`, but nothing
|
|
152
|
+
here enables the plugin or restarts the gateway. Enabling it is
|
|
153
|
+
`hermes plugins enable harnesstrim`, and that stays your command to run. No compatibility row ships
|
|
154
|
+
for Hermes because a row is the precondition for a *mutation*, and none is proposed.
|
|
155
|
+
|
|
156
|
+
Pi is read-only in both directions too: the adapter finds the HarnessTrim extension module in the
|
|
157
|
+
directories Pi auto-loads (`~/.pi/agent/extensions/` and `<project>/.pi/extensions/`) and verifies
|
|
158
|
+
the configuration, but nothing here installs it, and nothing here can say which mode it runs in —
|
|
159
|
+
the extension defaults to `dryrun` and only `HARNESSTRIM_MODE=active` in Pi's environment makes it
|
|
160
|
+
reduce. Installing it is `harnesstrim install pi --apply`, and that stays your command to run. No
|
|
161
|
+
compatibility row ships for Pi because a row is the precondition for a *mutation*, and none is
|
|
162
|
+
proposed.
|
|
163
|
+
|
|
164
|
+
RTK on OpenCode is detected and verified, not written: `rtk init -g --opencode` installs a plugin
|
|
165
|
+
module at `~/.config/opencode/plugins/rtk.ts`, and Token Harness reads that file rather than
|
|
166
|
+
producing it. Note that the plugin is inert under OpenCode Desktop — see
|
|
167
|
+
[docs/matrices.md](docs/matrices.md) for what was measured.
|
|
168
|
+
|
|
107
169
|
The generated compatibility tables, tested version ranges, platform coverage, and known
|
|
108
170
|
limitations are in [docs/matrices.md](docs/matrices.md).
|
|
109
171
|
|
|
@@ -148,13 +210,13 @@ If `corepack` is unavailable, install the pinned package manager with
|
|
|
148
210
|
|
|
149
211
|
### 2. Install or adopt RTK
|
|
150
212
|
|
|
151
|
-
|
|
213
|
+
When a reviewed compatibility row covers the installed versions, you normally do **not** install RTK yourself:
|
|
152
214
|
|
|
153
215
|
```sh
|
|
154
216
|
token-harness plan --harness claude --provider rtk
|
|
155
217
|
```
|
|
156
218
|
|
|
157
|
-
|
|
219
|
+
Once a matching compatibility row exists, if RTK is absent, the plan contains two actions:
|
|
158
220
|
|
|
159
221
|
1. install RTK through the selected package manager;
|
|
160
222
|
2. append one RTK entry to Claude Code's `PreToolUse` hook configuration.
|
|
@@ -205,9 +267,9 @@ token-harness doctor --provider rtk
|
|
|
205
267
|
|
|
206
268
|
### 3. Install or adopt HarnessTrim
|
|
207
269
|
|
|
208
|
-
With HarnessTrim
|
|
209
|
-
|
|
210
|
-
|
|
270
|
+
With HarnessTrim on `PATH`, `token-harness plan --harness claude` can install its Claude skills
|
|
271
|
+
without creating the competing Bash hook or reduce-pipe instruction. The invocation it delegates to
|
|
272
|
+
is:
|
|
211
273
|
|
|
212
274
|
```sh
|
|
213
275
|
harnesstrim install claude <project> --apply --no-hook --no-instructions
|
|
@@ -234,6 +296,19 @@ conflict instead of guessing an execution order. It never deletes the competing
|
|
|
234
296
|
HarnessTrim telemetry is opt-in in some adapters. Without a `.harnesstrim/metrics.jsonl` file,
|
|
235
297
|
verification can still inspect configuration, but `metrics` has no HarnessTrim events to import.
|
|
236
298
|
|
|
299
|
+
From `0.1.0`, HarnessTrim publishes a machine-readable capability declaration: the surfaces it
|
|
300
|
+
intercepts per coding agent, the flags that narrow an install, and the paths each install writes.
|
|
301
|
+
|
|
302
|
+
```sh
|
|
303
|
+
harnesstrim capabilities
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Detection reads that declaration and compares it against the one Token Harness records, so an
|
|
307
|
+
upstream change is reported rather than assumed compatible. A disagreement becomes a
|
|
308
|
+
`provider-capabilities-drift` warning naming both sides. A build older than the command cannot
|
|
309
|
+
answer; Token Harness then falls back to its own recorded declaration and reports nothing, because a
|
|
310
|
+
provider that cannot be asked must still be describable.
|
|
311
|
+
|
|
237
312
|
## The recommended operating workflow
|
|
238
313
|
|
|
239
314
|
### Step 1: diagnose
|
|
@@ -249,7 +324,9 @@ This answers:
|
|
|
249
324
|
- which agent configuration files exist;
|
|
250
325
|
- which provider is wired to which agent;
|
|
251
326
|
- whether Token Harness owns the integration or merely adopted it;
|
|
252
|
-
- whether a version, configuration file, or tool-family matcher needs attention
|
|
327
|
+
- whether a version, configuration file, or tool-family matcher needs attention;
|
|
328
|
+
- whether the installed provider's own capability declaration still agrees with the one Token
|
|
329
|
+
Harness records.
|
|
253
330
|
|
|
254
331
|
Common states:
|
|
255
332
|
|
|
@@ -369,6 +446,11 @@ The report keeps measurement types and units separate:
|
|
|
369
446
|
Token counts are never added to character counts, and estimated or counterfactual values are never
|
|
370
447
|
silently merged into an exact total.
|
|
371
448
|
|
|
449
|
+
The report covers one project: the one `--project` names, or the current directory. An operation a
|
|
450
|
+
provider recorded without a directory belongs to no project and is excluded, with a count reported
|
|
451
|
+
so the difference is reconcilable. When no project identity can be established the report says so
|
|
452
|
+
rather than presenting every project's events as one project's figures.
|
|
453
|
+
|
|
372
454
|
## Undoing changes
|
|
373
455
|
|
|
374
456
|
Choose the command based on what you want to undo:
|
|
@@ -492,8 +574,10 @@ Run `token-harness doctor`. The usual causes are:
|
|
|
492
574
|
- an existing user-managed integration already satisfies the target state;
|
|
493
575
|
- the safe profile excluded an overlapping provider.
|
|
494
576
|
|
|
495
|
-
RTK is
|
|
496
|
-
|
|
577
|
+
RTK is written only for Claude Code in 0.1.0. It claims OpenCode too, but the plan builder appends
|
|
578
|
+
a `hooks` entry, which is Claude Code's schema — OpenCode's integration is a plugin module, so an
|
|
579
|
+
OpenCode scope produces no action and the existing installation is adopted instead. A Codex-only
|
|
580
|
+
machine produces no RTK action at all.
|
|
497
581
|
|
|
498
582
|
### The plan is blocked by `exclusive-scope-contested`
|
|
499
583
|
|
|
@@ -519,6 +603,23 @@ Check all of the following:
|
|
|
519
603
|
|
|
520
604
|
An empty metrics report exits 0 because it is a valid observation, not a command failure.
|
|
521
605
|
|
|
606
|
+
### `doctor` or `status` reports `provider-capabilities-drift`
|
|
607
|
+
|
|
608
|
+
The installed provider's own capability declaration no longer agrees with the one Token Harness
|
|
609
|
+
records. The warning names both sides: what the recorded declaration claims, and what the installed
|
|
610
|
+
build reported. Nothing is modified, and the recorded declaration still drives planning.
|
|
611
|
+
|
|
612
|
+
Three disagreements are reported:
|
|
613
|
+
|
|
614
|
+
- a coding agent that Token Harness records a capability on is missing from the build's declaration;
|
|
615
|
+
- the reduction surface Token Harness records is absent from the surfaces the build reports;
|
|
616
|
+
- the build no longer covers a reviewed write-set path, or declares a path outside the reviewed
|
|
617
|
+
containment boundary.
|
|
618
|
+
|
|
619
|
+
The last one matters most before a delegated install. Rollback restores the reviewed boundary, so a
|
|
620
|
+
path outside it would survive a rollback. Re-review the write set at the installed version, or hold
|
|
621
|
+
at the reviewed one.
|
|
622
|
+
|
|
522
623
|
### A newer provider or agent version is reported
|
|
523
624
|
|
|
524
625
|
The tested ranges record versions actually exercised by this project. A newer version is reported
|
package/package.json
CHANGED
package/sbom.json
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
"type": "application",
|
|
8
8
|
"bom-ref": "token-harness",
|
|
9
9
|
"name": "token-harness",
|
|
10
|
-
"version": "0.1.
|
|
10
|
+
"version": "0.1.3",
|
|
11
11
|
"description": "One control plane for token-efficient coding agents.",
|
|
12
12
|
"licenses": [
|
|
13
13
|
{
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
"hashes": [
|
|
20
20
|
{
|
|
21
21
|
"alg": "SHA-256",
|
|
22
|
-
"content": "
|
|
22
|
+
"content": "d28d7044c0215e3471e990732b14275cca61e57a823e7a04a505748be73613c8"
|
|
23
23
|
}
|
|
24
24
|
]
|
|
25
25
|
},
|