token-harness 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +89 -12
- package/package.json +1 -1
- package/sbom.json +2 -2
- package/token-harness.mjs +1163 -90
package/README.md
CHANGED
|
@@ -32,6 +32,8 @@ neither installed nor configured by Token Harness.
|
|
|
32
32
|
| [LiteLLM](https://github.com/BerriAI/litellm) | Model routing, fallbacks, budgets, and usage telemetry | Not active — candidate |
|
|
33
33
|
| [RouteLLM](https://github.com/lm-sys/RouteLLM) | Route simpler requests to less expensive models | Not active — candidate |
|
|
34
34
|
| [vLLM Semantic Router](https://github.com/vllm-project/semantic-router) | Route by task, complexity, tools, and deployment locality | Not active — candidate |
|
|
35
|
+
| [Claude Code Router](https://github.com/musistudio/claude-code-router) | Route coding-agent requests across models and providers with effort-based rules and fallback chains | Not active — candidate · high priority |
|
|
36
|
+
| [LLMRouter](https://github.com/ulab-uiuc/LLMRouter) | Select the model by task complexity, cost, and quality across routing strategies | Not active — candidate · high priority |
|
|
35
37
|
| [Headroom](https://github.com/headroomlabs-ai/headroom) | Compress tool, MCP, file, and RAG payloads | Not active — candidate |
|
|
36
38
|
| [Context Mode](https://github.com/mksglu/context-mode) | Keep raw tool results outside model context | Not active — candidate |
|
|
37
39
|
| [LLMLingua](https://github.com/microsoft/LLMLingua) | Compress long prompts and context | Not active — candidate |
|
|
@@ -40,7 +42,9 @@ neither installed nor configured by Token Harness.
|
|
|
40
42
|
Candidate status means only that the project has identified a useful optimization layer. A tool
|
|
41
43
|
becomes active only after its installation, conflicts, rollback behavior, verification, and
|
|
42
44
|
metrics attribution have been implemented and tested. Token Harness never installs a candidate
|
|
43
|
-
merely because it is present on the machine.
|
|
45
|
+
merely because it is present on the machine. Rows marked `high priority` are the next intended
|
|
46
|
+
intake; the admission gates each one carries are recorded in
|
|
47
|
+
[docs/provider-landscape.md](docs/provider-landscape.md).
|
|
44
48
|
|
|
45
49
|
## Quick start
|
|
46
50
|
|
|
@@ -83,6 +87,35 @@ npx token-harness doctor
|
|
|
83
87
|
`npx` may download Token Harness into npm's cache, but it does not install or configure RTK,
|
|
84
88
|
HarnessTrim, or a coding agent.
|
|
85
89
|
|
|
90
|
+
### Managed compatibility rows
|
|
91
|
+
|
|
92
|
+
Token Harness changes a harness configuration only when a reviewed compatibility row covers the
|
|
93
|
+
exact provider version, harness version, platform, and configuration schema. Two rows ship, and each
|
|
94
|
+
names the recording it stands on:
|
|
95
|
+
|
|
96
|
+
| Provider | Harness | Platform | Tested versions | Tier |
|
|
97
|
+
| --- | --- | --- | --- | --- |
|
|
98
|
+
| RTK | Claude Code | Windows | rtk 0.44.0, Claude Code 2.1.220 | `canary` |
|
|
99
|
+
| HarnessTrim | Claude Code | Windows | harnesstrim 0.1.0, Claude Code 2.1.220 | `config-only` |
|
|
100
|
+
|
|
101
|
+
Everything else is refused, and that is the design rather than a gap: `doctor` detects and reports on
|
|
102
|
+
every supported platform, and only the *mutation* is narrower. An uncovered combination exits 9 and
|
|
103
|
+
the diagnostic names what is missing — the reviewed fixture, or the nearest row it does have.
|
|
104
|
+
|
|
105
|
+
What is not covered today, and why:
|
|
106
|
+
|
|
107
|
+
- **macOS and Linux.** No row on either. The recordings a row needs are states of a real machine, and
|
|
108
|
+
a fixture cannot be written from a machine nobody ran. On those platforms `plan` and `apply` refuse;
|
|
109
|
+
install the provider with its own installer and Token Harness will detect, verify, and measure it.
|
|
110
|
+
- **Codex and OpenCode.** Both providers are detected and adopted there, and neither is written:
|
|
111
|
+
RTK's plan builder produces a Claude-shaped hook list, and HarnessTrim's reviewed write set covers
|
|
112
|
+
Claude only. A row would admit a mutation that nothing proposes.
|
|
113
|
+
- **A newer Claude Code.** The range is a single observed version. `2.1.221` reads `unknown-newer` and
|
|
114
|
+
refuses rather than assuming it behaves like `2.1.220`.
|
|
115
|
+
|
|
116
|
+
The recordings are under `tests/fixtures/rows/`, one directory per row, each with a README stating
|
|
117
|
+
which stages exist and which do not.
|
|
118
|
+
|
|
86
119
|
## How the components fit together
|
|
87
120
|
|
|
88
121
|
There are three separate layers. Installing one does not automatically provide the others.
|
|
@@ -91,19 +124,24 @@ There are three separate layers. Installing one does not automatically provide t
|
|
|
91
124
|
| --- | --- | --- |
|
|
92
125
|
| Coding agent (harness) | Claude Code, Codex, OpenCode | You, using the agent's official installer |
|
|
93
126
|
| Token Harness | `token-harness` | You, from npm or this repository |
|
|
94
|
-
| Optimization provider | RTK, HarnessTrim |
|
|
127
|
+
| Optimization provider | RTK, HarnessTrim | Both can be installed by Token Harness where a compatibility row covers the combination; otherwise install them with their own installers and Token Harness detects and measures them |
|
|
95
128
|
|
|
96
129
|
Token Harness does not install Claude Code, Codex, or OpenCode. Install and run at least one of
|
|
97
130
|
them first so that `token-harness doctor` can detect it.
|
|
98
131
|
|
|
99
132
|
| Provider | Claude Code | Codex | OpenCode | Installed by Token Harness |
|
|
100
133
|
| --- | --- | --- | --- | --- |
|
|
101
|
-
| RTK | Configure, verify, and measure | Not managed |
|
|
102
|
-
| HarnessTrim | Claude skills only; no reducer hook or reduce-pipe instruction | Detect, adopt, verify, and measure | Detect, adopt, verify, and measure | **Yes**,
|
|
134
|
+
| RTK | Configure, verify, and measure | Not managed | Detect, adopt, verify, and measure | **Yes**, for the supported Claude Code path |
|
|
135
|
+
| HarnessTrim | Claude skills only; no reducer hook or reduce-pipe instruction | Detect, adopt, verify, and measure | Detect, adopt, verify, and measure | **Yes**, on a covered row — see above |
|
|
103
136
|
|
|
104
137
|
"Not managed" does not mean the upstream tool cannot support that agent. It means this release
|
|
105
138
|
does not claim ownership of that integration and will not modify it.
|
|
106
139
|
|
|
140
|
+
RTK on OpenCode is detected and verified, not written: `rtk init -g --opencode` installs a plugin
|
|
141
|
+
module at `~/.config/opencode/plugins/rtk.ts`, and Token Harness reads that file rather than
|
|
142
|
+
producing it. Note that the plugin is inert under OpenCode Desktop — see
|
|
143
|
+
[docs/matrices.md](docs/matrices.md) for what was measured.
|
|
144
|
+
|
|
107
145
|
The generated compatibility tables, tested version ranges, platform coverage, and known
|
|
108
146
|
limitations are in [docs/matrices.md](docs/matrices.md).
|
|
109
147
|
|
|
@@ -148,13 +186,13 @@ If `corepack` is unavailable, install the pinned package manager with
|
|
|
148
186
|
|
|
149
187
|
### 2. Install or adopt RTK
|
|
150
188
|
|
|
151
|
-
|
|
189
|
+
When a reviewed compatibility row covers the installed versions, you normally do **not** install RTK yourself:
|
|
152
190
|
|
|
153
191
|
```sh
|
|
154
192
|
token-harness plan --harness claude --provider rtk
|
|
155
193
|
```
|
|
156
194
|
|
|
157
|
-
|
|
195
|
+
Once a matching compatibility row exists, if RTK is absent, the plan contains two actions:
|
|
158
196
|
|
|
159
197
|
1. install RTK through the selected package manager;
|
|
160
198
|
2. append one RTK entry to Claude Code's `PreToolUse` hook configuration.
|
|
@@ -205,9 +243,9 @@ token-harness doctor --provider rtk
|
|
|
205
243
|
|
|
206
244
|
### 3. Install or adopt HarnessTrim
|
|
207
245
|
|
|
208
|
-
With HarnessTrim
|
|
209
|
-
|
|
210
|
-
|
|
246
|
+
With HarnessTrim on `PATH`, `token-harness plan --harness claude` can install its Claude skills
|
|
247
|
+
without creating the competing Bash hook or reduce-pipe instruction. The invocation it delegates to
|
|
248
|
+
is:
|
|
211
249
|
|
|
212
250
|
```sh
|
|
213
251
|
harnesstrim install claude <project> --apply --no-hook --no-instructions
|
|
@@ -234,6 +272,19 @@ conflict instead of guessing an execution order. It never deletes the competing
|
|
|
234
272
|
HarnessTrim telemetry is opt-in in some adapters. Without a `.harnesstrim/metrics.jsonl` file,
|
|
235
273
|
verification can still inspect configuration, but `metrics` has no HarnessTrim events to import.
|
|
236
274
|
|
|
275
|
+
From `0.1.0`, HarnessTrim publishes a machine-readable capability declaration: the surfaces it
|
|
276
|
+
intercepts per coding agent, the flags that narrow an install, and the paths each install writes.
|
|
277
|
+
|
|
278
|
+
```sh
|
|
279
|
+
harnesstrim capabilities
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
Detection reads that declaration and compares it against the one Token Harness records, so an
|
|
283
|
+
upstream change is reported rather than assumed compatible. A disagreement becomes a
|
|
284
|
+
`provider-capabilities-drift` warning naming both sides. A build older than the command cannot
|
|
285
|
+
answer; Token Harness then falls back to its own recorded declaration and reports nothing, because a
|
|
286
|
+
provider that cannot be asked must still be describable.
|
|
287
|
+
|
|
237
288
|
## The recommended operating workflow
|
|
238
289
|
|
|
239
290
|
### Step 1: diagnose
|
|
@@ -249,7 +300,9 @@ This answers:
|
|
|
249
300
|
- which agent configuration files exist;
|
|
250
301
|
- which provider is wired to which agent;
|
|
251
302
|
- whether Token Harness owns the integration or merely adopted it;
|
|
252
|
-
- whether a version, configuration file, or tool-family matcher needs attention
|
|
303
|
+
- whether a version, configuration file, or tool-family matcher needs attention;
|
|
304
|
+
- whether the installed provider's own capability declaration still agrees with the one Token
|
|
305
|
+
Harness records.
|
|
253
306
|
|
|
254
307
|
Common states:
|
|
255
308
|
|
|
@@ -369,6 +422,11 @@ The report keeps measurement types and units separate:
|
|
|
369
422
|
Token counts are never added to character counts, and estimated or counterfactual values are never
|
|
370
423
|
silently merged into an exact total.
|
|
371
424
|
|
|
425
|
+
The report covers one project: the one `--project` names, or the current directory. An operation a
|
|
426
|
+
provider recorded without a directory belongs to no project and is excluded, with a count reported
|
|
427
|
+
so the difference is reconcilable. When no project identity can be established the report says so
|
|
428
|
+
rather than presenting every project's events as one project's figures.
|
|
429
|
+
|
|
372
430
|
## Undoing changes
|
|
373
431
|
|
|
374
432
|
Choose the command based on what you want to undo:
|
|
@@ -492,8 +550,10 @@ Run `token-harness doctor`. The usual causes are:
|
|
|
492
550
|
- an existing user-managed integration already satisfies the target state;
|
|
493
551
|
- the safe profile excluded an overlapping provider.
|
|
494
552
|
|
|
495
|
-
RTK is
|
|
496
|
-
|
|
553
|
+
RTK is written only for Claude Code in 0.1.0. It claims OpenCode too, but the plan builder appends
|
|
554
|
+
a `hooks` entry, which is Claude Code's schema — OpenCode's integration is a plugin module, so an
|
|
555
|
+
OpenCode scope produces no action and the existing installation is adopted instead. A Codex-only
|
|
556
|
+
machine produces no RTK action at all.
|
|
497
557
|
|
|
498
558
|
### The plan is blocked by `exclusive-scope-contested`
|
|
499
559
|
|
|
@@ -519,6 +579,23 @@ Check all of the following:
|
|
|
519
579
|
|
|
520
580
|
An empty metrics report exits 0 because it is a valid observation, not a command failure.
|
|
521
581
|
|
|
582
|
+
### `doctor` or `status` reports `provider-capabilities-drift`
|
|
583
|
+
|
|
584
|
+
The installed provider's own capability declaration no longer agrees with the one Token Harness
|
|
585
|
+
records. The warning names both sides: what the recorded declaration claims, and what the installed
|
|
586
|
+
build reported. Nothing is modified, and the recorded declaration still drives planning.
|
|
587
|
+
|
|
588
|
+
Three disagreements are reported:
|
|
589
|
+
|
|
590
|
+
- a coding agent that Token Harness records a capability on is missing from the build's declaration;
|
|
591
|
+
- the reduction surface Token Harness records is absent from the surfaces the build reports;
|
|
592
|
+
- the build no longer covers a reviewed write-set path, or declares a path outside the reviewed
|
|
593
|
+
containment boundary.
|
|
594
|
+
|
|
595
|
+
The last one matters most before a delegated install. Rollback restores the reviewed boundary, so a
|
|
596
|
+
path outside it would survive a rollback. Re-review the write set at the installed version, or hold
|
|
597
|
+
at the reviewed one.
|
|
598
|
+
|
|
522
599
|
### A newer provider or agent version is reported
|
|
523
600
|
|
|
524
601
|
The tested ranges record versions actually exercised by this project. A newer version is reported
|
package/package.json
CHANGED
package/sbom.json
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
"type": "application",
|
|
8
8
|
"bom-ref": "token-harness",
|
|
9
9
|
"name": "token-harness",
|
|
10
|
-
"version": "0.1.
|
|
10
|
+
"version": "0.1.2",
|
|
11
11
|
"description": "One control plane for token-efficient coding agents.",
|
|
12
12
|
"licenses": [
|
|
13
13
|
{
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
"hashes": [
|
|
20
20
|
{
|
|
21
21
|
"alg": "SHA-256",
|
|
22
|
-
"content": "
|
|
22
|
+
"content": "8e2f96efbb15fd8aec72a98ae3b62d128265abd779e9f911c8c0d4bf024adfb2"
|
|
23
23
|
}
|
|
24
24
|
]
|
|
25
25
|
},
|