@bglocation/tune-context 2.5.0 → 3.0.0

package/.claude-plugin/marketplace.json

@@ -10,7 +10,7 @@
       "name": "tune-context",
       "source": "./",
       "description": "Context-efficiency configurator for Claude Code: doctrine skills (token-efficiency, caveman, cdd, phase-workflow) plus a tune-context configurator that scaffolds a lean CLAUDE.md, subagents, settings and hooks.",
-      "version": "2.5.0",
+      "version": "3.0.0",
       "author": {
         "name": "Szymon Walczak",
         "url": "https://bglocation.dev"

package/.claude-plugin/plugin.json

@@ -2,7 +2,7 @@
   "name": "tune-context",
   "displayName": "tune-context",
   "description": "Context-efficiency configurator for Claude Code: doctrine skills (token-efficiency, caveman, cdd, phase-workflow) plus a tune-context configurator that scaffolds a lean CLAUDE.md, subagents, settings and hooks.",
-  "version": "2.5.0",
+  "version": "3.0.0",
   "author": {
     "name": "Szymon Walczak",
     "url": "https://bglocation.dev"

package/CHANGELOG.md

@@ -5,6 +5,85 @@ All notable changes to `@bglocation/tune-context` are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.0.0] — 2026-10-15
+
+EPIC-010 — the meter of the cost measurement comes from the same installed
+version as the switch, and every artifact names the code that computed it.
+**The major bump is for one deliberately breaking change** — the eval data
+contract (see Changed): rows without a meter stamp and `analyze@1` aggregates
+are refused. Everything else is additive.
+
+### Added
+
+- **`tune-context eval <step>`** (TASK-095) — `prompt-metrics`, `analyze`,
+  `verdict`, `report`, `adoption-report`, named after their modules, no aliases,
+  from any directory. Until now the meter existed only as `node eval/<file>.mjs`,
+  a relative path that resolves in a checkout of this repo and nowhere else, so
+  the switch (from `PATH`) and the meter (from `cwd`) could be two versions.
+  Everything after the step goes verbatim to the step's own parser:
+  `tune-context eval prompt-metrics --help` prints the step's usage. The two
+  doors print the same bytes, held to a baseline written before the refactor.
+- **`tune-context eval verdict`** (TASK-096) — the verdict as JSON on stdout (the
+  object the report embeds) and one line per gate on stderr, ending in the
+  finding. Exit 0 for every verdict, a withheld one included. `verdict` and
+  `report` read their input through one function, so the page cannot accept what
+  the verdict refuses.
+- **The meter stamp** (TASK-097) — rows carry `meter_version`, the aggregate
+  `analyzed_by`, the verdict `judged_by`: the bare release number for an npm
+  install, `<version>+<sha12>` for a checkout, `.dirty` for uncommitted changes in
+  `eval/ cli/ hooks/ bin/ package.json`, `+unknown` where git could not answer.
+  A version number alone would not do: under `npm link` a fix committed before
+  the release carries the number of the bug it fixes. `prompt-metrics` prints
+  `meter <stamp>` as the first line of its summary.
+- **A seventh verdict gate, `meter_mixed`** (frozen by the PO on 2026-10-15) — the
+  whole chain (paired rows, `analyzed_by`, `judged_by`) must carry exactly one
+  stamp that names code; `.dirty`, `+unknown`, a second stamp or a step with none
+  hold the verdict, and the reason names the step that brought the stranger.
+  The six earlier thresholds are untouched.
+- **`tune-context experiment archive --out <dir>`** (TASK-098) — the series'
+  transcripts (every session on an `exp/…` branch) and the ledger, copied out of
+  reach of Claude Code's retention, with `manifest.json` and `SHA256SUMS`
+  (`shasum -a 256 -c`). Append-only: a grown file replaces its copy, any other
+  change refuses the run. A destination inside a git work tree that git does not
+  ignore is refused — a transcript is the whole conversation.
+- **`--ledger <path>`** for `prompt-metrics` and `analyze`, so an archive
+  recounts as `branch+ledger` on any machine.
+
+### Changed
+
+- **The eval data contract — deliberately breaking** (EPIC-010, PO decision 4).
+  Schemas are now `tune-context/analyze@2`, `tune-context/verdict@2` and
+  `tune-context/verdict-thresholds@2`. `analyze` refuses a whole input containing
+  a single row without `meter_version` (exit 1, nothing on stdout, file and line
+  on stderr), and `verdict`/`report` refuse an `analyze@1` aggregate. Scripts
+  that feed rows or aggregates written by an earlier version get exit 1. That is
+  the point: unstamped data cannot say which meter computed it, and the way back
+  to it is recomputing the transcripts — from an `experiment archive`, with the
+  current version or with `npx -p @bglocation/tune-context@<version>`.
+- **The cost evaluation runbook and procedure** run the series from a registry
+  release (`npm link` only for an unreleased fix, from a clean commit), record
+  the meter stamp before the first pair, and archive after every arm.
+- **Skills** — `token-efficiency` and `tune-context` point at
+  `tune-context eval adoption-report` instead of `node eval/…`, which never
+  resolved outside this repo, and `tune-context` names
+  `tune-context eval prompt-metrics` as the source of token cost, where it said
+  only OTEL with a collector had it. Skills are copies: **re-run
+  `tune-context init` after upgrading** to get them.
+- **`adoption-report` points at `tune-context eval prompt-metrics`** (TASK-100) —
+  in `--help` and in the report's closing note on cost, instead of
+  `eval/prompt-metrics.mjs`, a path that resolves only in a checkout of this repo.
+  Likewise `analyze` — its `--help` and the last line of its summary — points at
+  `tune-context eval verdict` instead of a task number that is not in the package.
+
+### Fixed
+
+- **A reader that leaves early is a normal end for every eval step** — `report`
+  writing a long page, or `analyze` an aggregate of many tasks, into `| head`
+  ended in an `EPIPE` stacktrace; only `prompt-metrics` and `verdict` had a
+  guard. One guard now, called by each step.
+- **`report --out` into a directory that does not exist** is one line and exit 1,
+  not a stacktrace.
+
 ## [2.5.0] — 2026-09-03
 
 ### Fixed
@@ -655,6 +734,7 @@ action needed on upgrade.
   (`explore-cheap`, `reviewer`), the PreCompact / SessionStart / PostToolUse hook
   suite, and the adoption report (`eval/adoption-report.mjs`).
 
+[3.0.0]: https://gitlab.com/bglocation/tune-context/-/tags/v3.0.0
 [2.5.0]: https://gitlab.com/bglocation/tune-context/-/tags/v2.5.0
 [2.4.0]: https://gitlab.com/bglocation/tune-context/-/tags/v2.4.0
 [2.3.1]: https://gitlab.com/bglocation/tune-context/-/tags/v2.3.1

package/README.md

@@ -6,7 +6,8 @@ scaffolds a cost-optimal, high-quality setup (lean CLAUDE.md + skills + subagent
 + settings + hooks + MCP).
 
 **Self-contained** — zero dependency on any host repo. Ships two ways: a Claude
-Code **plugin** and an **npm CLI** (`tune-context init`), both under
+Code **plugin** and an **npm CLI** (`tune-context init`; plus `verify`, and
+`experiment` and `eval` for measuring the doctrine's cost), both under
 `@bglocation`. Incubated inside the [`code-search-mcp`](https://gitlab.com/bglocation/code-search-mcp)
 project, now extracted to its own repository.
 
@@ -16,9 +17,9 @@ tune-context/
 ├── skills/            # token-efficiency, caveman, cdd, phase-workflow, tune-context (+ templates/)
 ├── agents/            # explore-cheap, reviewer (model right-sizing)
 ├── hooks/             # PreCompact state-doc, SessionStart re-inject, adoption log, context reminder + hooks.json
-├── bin/ + cli/        # `tune-context init` — deterministic detect → generate → verify
-├── eval/              # adoption-report.mjs (the rag-usage analog), prompt-metrics.mjs, pricing.mjs, arm.mjs, analyze.mjs, verdict.mjs, report.mjs
-└── scripts/           # verify-pack, smoke-init, sync-tune-context (dev/maintenance)
+├── bin/ + cli/        # `tune-context init | verify | experiment | eval` — deterministic, no model in the loop
+├── eval/              # adoption-report.mjs (the rag-usage analog) + mcp-probe.mjs, prompt-metrics.mjs, pricing.mjs, arm.mjs, analyze.mjs, verdict.mjs, report.mjs
+└── scripts/           # verify-pack, verify-plugin, smoke-init, sync-tune-context (dev/maintenance)
 ```
 
 ## Quick start
@@ -165,7 +166,7 @@ claude --plugin-dir /path/to/tune-context                       # or a local che
 ```
 
 The plugin manifest (`.claude-plugin/plugin.json`) and hook wiring
-(`hooks/hooks.json`, via `${CLAUDE_PLUGIN_ROOT}`) register the four hook events
+(`hooks/hooks.json`, via `${CLAUDE_PLUGIN_ROOT}`) register the five hook events
 automatically; `bin/tune-context` is added to `PATH` while the plugin is enabled.
 The marketplace catalog is [`.claude-plugin/marketplace.json`](.claude-plugin/marketplace.json).
 
@@ -246,7 +247,7 @@ by `tune-context init`), not here.
 ## Measuring adoption (does the doctrine actually get used?)
 `hooks/adoption-log.mjs` logs one JSONL line per event to
 `~/.claude/tune-context/adoption.jsonl` by default (under `CLAUDE_CONFIG_DIR`, the
-override's `tune-context/adoption.jsonl` — see above), and `eval/adoption-report.mjs`
+override's `tune-context/adoption.jsonl` — see above), and `tune-context eval adoption-report`
 turns that into a report — code search (semantic RAG vs the Grep tool, with
 shell-plumbing greps kept out of the RAG share), doctrine skills, subagents, and
 re-read churn — closing with an *earned* verdict: withheld below a PO-frozen floor of
@@ -321,10 +322,10 @@ instead of printing a number the doubling would have faked.
    `~/.claude/tune-context/adoption.jsonl` by default (created on first write; under
    `CLAUDE_CONFIG_DIR` it's the override's `tune-context/adoption.jsonl` instead).
 
-4. **Read the report** any time, from this checkout (or the installed package):
+4. **Read the report** any time, from any directory:
    ```bash
-   node eval/adoption-report.mjs            # default log location
-   node eval/adoption-report.mjs --log <path>
+   tune-context eval adoption-report            # default log location
+   tune-context eval adoption-report --log <path>
    ```
 
 **Stop / opt out:** delete the `PostToolUse` + `SubagentStart` entries from
@@ -340,7 +341,45 @@ contracts-first edits, delegation — is only partly visible too: no `Skill` cal
 but its trace shows up in the report's edit:read ratio, so the verdict never
 reads zero `Skill` calls as "unused."
 
-### Per-prompt metrics from transcripts (`eval/prompt-metrics.mjs`)
+### The meter, from any directory (`tune-context eval`)
+
+The cost measurement has two halves: the **switch** (`tune-context experiment`,
+below) and the **meter** — `prompt-metrics → analyze → verdict → report`. Both
+are one command, so both come from one installed version:
+
+```bash
+tune-context eval --help                    # the steps
+tune-context eval <step> --help             # that step's flags
+```
+
+The step names are the module names (`eval/<step>.mjs`), one to one, no aliases.
+Everything after the step belongs to the step — `tune-context eval
+prompt-metrics --help` prints the step's usage, not the CLI's.
+
+**Every artifact names the code that computed it.** Rows carry `meter_version`,
+the aggregate `analyzed_by`, the verdict `judged_by` — a *meter stamp*, which
+identifies code rather than a release number:
+
+| stamp | when |
+| --- | --- |
+| `3.0.0` | an npm install — immutable on the registry, rebuilt on demand with `npx -p @bglocation/tune-context@3.0.0` |
+| `3.0.0+0123456789ab` | a checkout of this repo, clean in `eval/ cli/ hooks/ bin/ package.json` |
+| `3.0.0+0123456789ab.dirty` | the same with uncommitted changes there (untracked files included) |
+| `3.0.0+unknown` | a checkout git could not read |
+
+A release number alone is not enough: under `npm link` a fix committed before
+the release carries the same number as the bug it fixes, which is how EPIC-009's
+verdict ended up computed by a meter nobody could name. Rows without a stamp —
+anything from before this contract — are refused, not converted: the way back to
+old data is recomputing the transcripts (see
+[`experiment archive`](#keeping-the-evidence-tune-context-experiment-archive)).
+
+*Developer path, from a checkout of this repo only:* `node eval/<step>.mjs` is
+the same code behind a second door — same parser, same bytes on stdout — and
+stamps `<version>+<sha>` (`.dirty` while you have uncommitted changes, which
+holds every verdict by design). A measurement runs through `tune-context eval`.
+
+### Per-prompt metrics from transcripts (`tune-context eval prompt-metrics`)
 
 The adoption report reads *our* hook log. `prompt-metrics` reads the transcripts
 **Claude Code writes itself** (`<config dir>/projects/<slug>/<session>.jsonl`) and
@@ -348,11 +387,14 @@ emits one JSONL row per prompt — time, tokens split by cache class, model,
 effort, tools, delegations, and cost in dollars (`eval/pricing.mjs`):
 
 ```bash
-node eval/prompt-metrics.mjs                              # every transcript
-node eval/prompt-metrics.mjs --transcript <path.jsonl>    # one session
-node eval/prompt-metrics.mjs --dir <dir> > prompts.jsonl  # JSONL on stdout
+tune-context eval prompt-metrics                              # every transcript
+tune-context eval prompt-metrics --transcript <path.jsonl>    # one session
+tune-context eval prompt-metrics --dir <dir> > prompts.jsonl  # JSONL on stdout
 ```
 
+The first line of the summary on stderr is `meter <stamp>` — the line a
+measurement series copies into its run journal before the first pair.
+
 Rows go to stdout, a run summary to stderr — so a pipe stays clean. Two things
 that source makes possible and a hook cannot: it works for **past** sessions, and
 it works when tune-context is switched **off** (a disabled hook records nothing,
@@ -456,17 +498,67 @@ The measurement protocol that uses this — paired runs on `exp/<task>/<arm>`
 branches, frozen prompt script, verdict gates — is
 [COST_EVALUATION_PROCEDURE.md](COST_EVALUATION_PROCEDURE.md).
 
-### Per-arm aggregate (`eval/analyze.mjs`)
+### Keeping the evidence (`tune-context experiment archive`)
+
+Claude Code deletes transcripts after `cleanupPeriodDays` (30 by default). A
+meter bug found after that cannot be fixed by recomputing — it voids the series.
+So the series' transcripts are archived **after every arm**, together with the
+ledger that assigns them to arms:
+
+```bash
+tune-context experiment archive --out ~/tune-context-archive/series-1
+```
+
+- **What goes in:** every transcript with at least one entry on an `exp/…`
+  branch — including branches that only mimic the convention, so a recount can
+  reject them by name — and the ledger, in the config dir's own layout, plus
+  `manifest.json` (hash, size, branches, Claude Code versions per file) and
+  `SHA256SUMS`, checkable with `shasum -a 256 -c SHA256SUMS`. Ordinary work is
+  not archived: it is not evidence, and it is your whole history.
+- **Append-only.** An unchanged file is skipped; a file that grew (a resumed
+  session, a longer ledger) replaces its copy and is named; any other change
+  refuses the whole run and leaves the archive as it was. A file Claude Code has
+  since deleted stays — that is what the archive is for. Re-running costs
+  nothing.
+- **A transcript is the whole conversation** — contents of the files read,
+  command output, possibly secrets. A destination inside a git work tree that
+  git does not ignore is refused, with no flag to override it: add it to
+  `.gitignore` or archive outside any repository.
+
+Recomputing from the archive works after the originals are gone. `--ledger`
+points both steps at the archived ledger — without it the rows come out
+`exp_arm_source: branch`, which the paired pool does not accept:
+
+```bash
+tune-context eval prompt-metrics --dir ~/tune-context-archive/series-1/projects \
+  --ledger ~/tune-context-archive/series-1/tune-context/experiment.jsonl > rows.jsonl
+tune-context eval analyze --rows rows.jsonl \
+  --ledger ~/tune-context-archive/series-1/tune-context/experiment.jsonl > aggregate.json
+```
+
+To recompute with the meter that computed the original rows instead of the
+current one, run every step with that version:
+`npx -p @bglocation/tune-context@<version> tune-context eval …`.
+
+### Per-arm aggregate (`tune-context eval analyze`)
 
 Rows are data, not an answer. `analyze` turns a stream of them into one
 aggregate per arm — and stops one step short of ruling which arm won, because
 the thresholds that would do the ruling are frozen by the PO, not by the model:
 
 ```bash
-node eval/prompt-metrics.mjs > rows.jsonl
-node eval/analyze.mjs --rows rows.jsonl > aggregate.json   # summary on stderr
+tune-context eval prompt-metrics > rows.jsonl
+tune-context eval analyze --rows rows.jsonl > aggregate.json   # summary on stderr
+tune-context eval prompt-metrics | tune-context eval analyze > aggregate.json   # or through a pipe
 ```
 
+- **No stamp, no aggregate.** One row without `meter_version` refuses the
+  **whole** input — exit 1, nothing on stdout, file and line of the first three
+  on stderr. A file mixing old and new rows is a pasted-together input, and
+  skipping the old ones would hand the verdict an aggregate that looks complete.
+  Rows from *different* meters are aggregated (the per-pair costs stay visible
+  for planning a repeat); the verdict holds on them.
+
 - **Two populations, never one number.** The paired measurement (branch
   `exp/<task>/<arm>`, everything else held constant) and ordinary daily work
   (attributed by the ledger, nothing held constant) come out as two separate
@@ -491,23 +583,47 @@ node eval/analyze.mjs --rows rows.jsonl > aggregate.json   # summary on stderr
   the effect being looked for. The per-pair, per-arm cost is carried so the two
   can be separated arithmetically (procedure §11b).
 
-### The report (`eval/report.mjs`)
+### The verdict as data (`tune-context eval verdict`)
 
-One HTML file — data embedded as JSON, charts as inline SVG, no network and no
-build step, so it opens from `file://` on a machine with nothing installed:
+May this run say "one arm is cheaper"? Seven gates answer, each with its own
+frozen threshold, and every gate is evaluated every time — a run that fixes one
+problem and reruns should not pay a run to learn about the second:
 
 ```bash
-node eval/analyze.mjs --rows rows.jsonl > aggregate.json
-node eval/report.mjs --aggregate aggregate.json --out report.html
-node eval/report.mjs --aggregate aggregate.json --rejected-runs journal.json --out report.html
+tune-context eval verdict --aggregate aggregate.json --rejected-runs journal.json > verdict.json
 ```
 
+stdout is the verdict as JSON — the same object the report embeds — and stderr
+one line per gate (measured against threshold, and why it holds) ending in the
+finding: `withheld: N gate(s) hold — …`, `no-difference` or `cheaper-arm: <arm>`.
+Exit 0 for every verdict, a withheld one included; exit 1 only for input that
+cannot be read.
+
 `--rejected-runs` carries what the run journal recorded as §7 invalidations
 (`[{ session, task, arm, prompts, reason }]`). **Omitting it is not the same as
 passing `[]`**: the first says nobody asked the journal, and the verdict is held
 for exactly that reason — a rejection share whose denominator is unknown is a
 quietly dropped sample wearing a passing gate.
 
+The seventh gate, `meter_mixed` (frozen by the PO on 2026-10-15), passes only
+when the whole chain — the paired rows, `analyzed_by`, `judged_by` — carries
+exactly one stamp, and one that names code: not `.dirty`, not `+unknown`. A
+series therefore runs from a release, or from a clean commit.
+
+### The report (`tune-context eval report`)
+
+One HTML file — data embedded as JSON, charts as inline SVG, no network and no
+build step, so it opens from `file://` on a machine with nothing installed:
+
+```bash
+tune-context eval report --aggregate aggregate.json --out report.html
+tune-context eval report --aggregate aggregate.json --rejected-runs journal.json --out report.html
+```
+
+It reads its input exactly as `eval verdict` does — one reader, so the page can
+never accept what the verdict refuses — and names the stamp of every step of the
+chain in its header. `--rejected-runs` means the same as above.
+
 - **A withheld verdict is the page, not a footnote.** When a gate holds, the
   arm-comparison chart is *not drawn at all* — not greyed out, not captioned. Two
   bars side by side assert "this is a comparison" more loudly than any caption

package/bin/tune-context

@@ -8,4 +8,4 @@
 // process.argv.
 import { main } from './tune-context.mjs';
 
-main();
+await main();