loopmath 0.3.0__tar.gz → 0.3.2__tar.gz

{loopmath-0.3.0 → loopmath-0.3.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: loopmath
-Version: 0.3.0
+Version: 0.3.2
 Summary: Plan agent workflows, record how they went, and learn from your own Claude Code and Codex logs
 Author: QKV Labs
 License-Expression: MIT
@@ -174,14 +174,19 @@ The workflow builder is the one page that is served: `loopmath builder --type bu
 | Group | Commands |
 |---|---|
 | Plan | `task-types`, `workflows` (list, show, validate, diff), `recommend`, `builder` |
-| Record | `run` (start, record, import, attempt, artifact, finish, sessions), `outcome`, `budget`, `config` (get, set) |
+| Record | `run` (start, record, import, attempt, artifact, finish, sessions), `outcome`, `budget`, `config` (get, set, unset) |
 | Learn | `fit`, `onboard`, `share`, `prior` |
 | View | `runs`, `posterior`, `status`, `report`, `doctor`, `verify-receipts` |
+| Harness testing | `check-change`, `gate` (experimental), `screen` (experimental) |
+| Task subsets | `tasks pick` (experimental) |
+| Evolution loops | `adapt openevolve` (experimental), `adapt shinkaevolve` (experimental), `recommend --role` (experimental) |
 | Skill | `skill` (install, uninstall, show) |
 | OCP | `ocp` (validate, migrate) |
-| Log analysis | `analyze`, `graph`, `adapt`, `prices`, `validate-prices`, `scan` |
+| Log analysis | `analyze`, `graph`, `adapt`, `prices`, `validate-prices` |
 
-Each command prints a short summary by default. With `--json`, it prints exactly one JSON object instead. One exception: `loopmath adapt harness2` and `adapt harness-runs` without `--out` print the run documents on stdout, one per line, so their `--json` summary goes to stderr. Exit codes: 0 ok, 1 user error, 2 not found, 3 not implemented, 4 store locked, 5 no fit yet. For flags, `loopmath <command> --help` is the source of truth. Commands, pages and the `runs` table show times in your local time with the zone; the store and `--json` keep UTC. A cost under a cent shows one significant digit (`$0.004`), and an exact zero is `$0`.
+These are experimental: `gate`, `screen`, `tasks pick`, the harness-part estimates (`recommend --harness-variants` and `--harness`, `posterior --level harness` and its `factors`), `recommend --role`, `adapt harness2`, `adapt plugin-eval`, `adapt openevolve`, `adapt shinkaevolve` and the ADE adapters (`adapt NAME`). Each works and has tests, but has had little use on real work so far, and its output may change. Their help says so too. The gate at DSPy's GEPA and MIPROv2 accept step (`loopmath.integrations.dspy`, Python) is experimental too: it is tested against fakes written from their source, not yet with DSPy installed, and its guide says so. So is the opt-in cost model for arms that mix models, `loopmath.compare.model.COST_MODEL = "sw"`, a Python setting with no help of its own; the [changelog](CHANGELOG.md) describes it.
+
+Each command prints a short summary by default. With `--json`, it prints exactly one JSON object instead. One exception: `loopmath adapt harness2`, `adapt harness-runs`, `adapt plugin-eval`, `adapt openevolve` and `adapt shinkaevolve` without `--out` print the run documents on stdout, one per line, so their `--json` summary goes to stderr. Exit codes: 0 ok, 1 user error, 2 not found, 3 not implemented, 4 store locked, 5 no fit yet. For flags, `loopmath <command> --help` is the source of truth. Commands, pages and the `runs` table show times in your local time with the zone; the store and `--json` keep UTC. A cost under a cent shows one significant digit (`$0.004`), and an exact zero is `$0`.
 
 A workflow is a TOML file. `loopmath workflows list` shows the catalog and yours, which live in `~/.loopmath/workflows/`. It also lists the shapes loopmath composes from catalog pieces, such as `team`; `loopmath workflows show team` says what one is and that `recommend` does not offer it. This one implements, then sends a rejected change back once or twice:
 
@@ -255,21 +260,44 @@ A run can record the harness it ran with: the prompt files, skills, plugins, MCP
   `cost` is the multiplier on an attempt's dollars when the part is present; `success` is percentage points.
 - `loopmath recommend --harness-variants` prices the trees your runs used most, and `--harness PATH` prices one more, against the bare harness label on the same draws.
 
-An estimate is the change when a named part is present, pooled over its versions. It is not the effect of one edit, and a part on every run cannot be told apart from its harness. The cost ranges met their stated coverage in simulation. **The success ranges did not:** they pull effects toward 0, and their 80% ranges held 60% of nonzero effects. So they are marked `calibrated: false`, and so are the success-side numbers of a priced variant. With no record anywhere, every number is what it was before. The record, the digest, the factor map and the limits are in [`spec/harness.md`](spec/harness.md).
+An estimate is the change when a named part is present, pooled over its versions. It is not the effect of one edit, and a part on every run cannot be told apart from its harness. The cost ranges met their stated coverage in simulation. **The success ranges did not:** they pull effects toward 0, and their 80% ranges held 60% of nonzero effects. So they are marked `calibrated: false`, and so are the success-side numbers of a priced variant, and the harness-part estimates are experimental. With no record anywhere, every number is what it was before. The record, the digest, the factor map and the limits are in [`spec/harness.md`](spec/harness.md).
 
 ## Testing a harness change
 
 These commands are for changing the harness your agents run with and finding out whether the change paid off.
 
 - **Harness parts and `--harness`.** Record the tree each run used (`run finish --harness auto` or `PATH`), and the fit learns each named part's effect, as in the section above. `recommend --harness-variants` prices the trees your runs used on the configurations those runs used, and on the recommendation's own when their harness has data on the parts; `--harness PATH` prices one more tree. Where a tree is not priced, `not_priced` in `--json` says why. The record, the factor map and the limits are in [`spec/harness.md`](spec/harness.md).
-- **`adapt harness-runs` and `adapt harness2`.** These bring in runs another tool made. `loopmath adapt harness-runs FILE --out DIR` reads a `loopmath.harness-runs/0.1` file, one JSON line per rollout, as an auto-research loop, a plugin eval suite or a hand A/B writes it. `loopmath adapt harness2 RUNS --out DIR` reads a harness2 runs folder. Then `loopmath run import DIR --finish` stores the runs. The row format, outcomes, costs and worked examples are in [`spec/harness-runs.md`](spec/harness-runs.md).
+- **`adapt harness-runs` and `adapt harness2`.** These bring in runs another tool made; `adapt harness2` is experimental, since its input format is read from harness2's code and it has not yet read a real runs folder. `loopmath adapt harness-runs FILE --out DIR` reads a `loopmath.harness-runs/0.1` file, one JSON line per rollout, as an auto-research loop, a plugin eval suite or a hand A/B writes it. `loopmath adapt harness2 RUNS --out DIR` reads a harness2 runs folder. Then `loopmath run import DIR --finish` stores the runs. The row format, outcomes, costs and worked examples are in [`spec/harness-runs.md`](spec/harness-runs.md).
 - **`check-change`.** It decides whether a change beats what it replaces, on paired runs:
-  - `check-change start --base LABEL|PATH --cand LABEL|PATH --n N` plans the pairs and each pair's arm order;
-  - `check-change status` gives the comparison, the verdict once there is one (adopt or reject) and the next pair worth running;
+  - `check-change start --base LABEL|PATH --cand LABEL|PATH --n N` plans the pairs and each pair's arm order; `--base-set PIECE=HARNESS:MODEL:EFFORT` and `--cand-set` give each arm a setting, as `run record --set` takes it, for a model switch on one harness: the arm's runs take its set, a run with another is refused, and `status` names the models each arm ran and any run whose log ran another;
+  - `check-change status` gives the comparison, the verdict once there is one (adopt or reject) and the next pair worth running. A check that reaches its cap or spends its budget before the evidence decides keeps the base and says `not decided`. The next pair's value and price are shown at each look and stop nothing; `--value-stop` also stops a check, not decided, when the next pairs are certified not worth their price (every plan's net below 0 by more than two standard errors, or no look left can change the action), from 4 scored pairs on (`--min-pairs K`). Use it when a pair is dear against what an adoption is worth over `--horizon-runs`: in simulated checks at the default 50 future runs it saved 6 to 16 pairs a check but dropped 26 to 40 percent of the adoptions of real cuts, and at 500 it dropped none (`check-change start --help`); `--pair-price 0` compares the value with 0 for runs on a subscription, and the budget still counts the runs' recorded dollars; `--json` keeps `verdict: reject` there, with `comparison.stop` and `comparison.decided: false`;
   - `check-change from-runs --base-label SEL --cand-label SEL` compares finished runs already in the store, picked by a run label, a configuration id, a harness digest (a prefix of one digest will do) or `arm:NAME`.
 
-  `--rule` names what counts as accepted, and `loopmath check-change --help` lists the rest. A run with no dollar cost leaves the decision to the outcomes alone, and the cost reads unknown; a budget or `--objective cpar` then refuses, naming the runs without one.
-- **`gate`.** For a loop that adopts changes on its own, it answers: adopt the candidate, reject it and keep the base, or replicate N more pairs first. `loopmath gate --base SEL --cand SEL` reads the finished runs in the store, or `--runs FILE` reads `loopmath.harness-runs/0.1` rows; an arm is a harness digest (`sha256:<hex>` or `digest:sha256:<hex>`, a prefix of one digest will do), a configuration id, or `arm=NAME` or `arm:NAME`; `check-change from-runs` takes the same forms. `--rule` takes the built-in rules or a rule defined under `[rules.NAME]` in the store's config, which the gate reads as `check-change` does, also with `--runs`. The comparison of the paired runs is `check-change`'s, and the gate adds the loop's policy to it: no adoption before `--r-min` pairs, looks at 1, 2, 4 and so on pairs up to `--r-max`, a `--floor` (the smallest gain worth adopting, 0 or more; under an acceptance rule it applies to the cut in cost per accepted result, a fraction, or to the gain in the chance of acceptance when a cost is unknown), and the dollars, `--budget-usd` for both arms (0 buys no further pair) and `--max-cost-ratio`. With no evidence, an arm that matches no run or a rule that scores no pair, it exits 1 and names the arms or the rules that are there. Both dollar flags need every run's cost and refuse, naming the runs without one; with neither flag, a run with no recorded cost leaves the decision to the outcomes alone, and the cost reads unknown, never $0. When nothing resolves within the cap or the budget, it keeps the base and says where it stopped. In Python, `loopmath.gate.decide(base_runs, cand_runs, GatePolicy(...))` gives the same answer and writes nothing; `loopmath gate --help` lists the rest.
+  `--rule` names what counts as accepted, and `loopmath check-change --help` lists the rest. A run with no dollar cost leaves the decision to the outcomes alone, and the cost reads unknown; a budget or `--objective cpar` then refuses, naming the runs without one. When an arm's runs mix a cheap model and a dear one, the default cost model can put the dearer arm's cost per run below the cheaper arm's. In Python, the experimental setting `loopmath.compare.model.COST_MODEL = "sw"` reads such costs for `compare.paired` and `gate.decide`; the [changelog](CHANGELOG.md) gives its limits.
+- **`screen` (experimental).** It tests many harness edits at once. Put each edit in its own subfolder of an edits folder, laid over a base harness tree: `PATH` copies a file, `PATH.append` appends to one, `PATH.remove` removes lines. Then:
+  - `screen start --base DIR --edits DIR --tasks FILE.jsonl --rule tests` plans one fold-over pair per task: arm A runs the edits its row turns on, arm B the others, so task difficulty cancels in the pair and every session informs every edit. 12, 16, 20, 24 or 32 pairs for up to one edit fewer give orthogonal rows;
+  - `screen next` prints the next pair, with both arms' trees built in the store; `run record --screen SCR --arm A|B` records a run into it;
+  - `screen status` ranks the edits on success and cost with 80% ranges and P(helps), names the predicted-best subset with its predicted cut in cost per accepted result (often a subset no arm ran), and prices the next pair. `--on EDIT,EDIT` predicts any subset;
+  - `screen apply --best --out DIR` writes the base with the best edits, ready for a `check-change` against the base.
+
+  The model is a Bayesian linear model of each pair's difference, on success and on log cost, with its noise and effect scales weighed over a grid rather than plugged in. A finished orthogonal screen weighs two views, a few edits matter (sparse) or all matter a little (dense), by how well each explains the pairs, and `best.by_view` shows both. On a dozen pairs each edit's effect is shrunk toward 0 (a large one by about half with 8 edits in 12 pairs), but in simulations at that size, of 1,000 screens drawn from effect settings no tuning used, 964 picked a best subset and its 80% range held the true cut in 812 (84%), a few points high where the true cut was small and low where it was large; the other 36 picked no edit, which predicts no change. The prediction is against the base, which no arm runs, so an effect of having the edited files at all is not in it. The [screen guide](docs/screen.md) has a worked example and the limits.
+- **`gate` (experimental).** For a loop that adopts changes on its own, it answers: adopt the candidate, reject it and keep the base, or replicate N more pairs first. `loopmath gate --base SEL --cand SEL` reads the finished runs in the store, or `--runs FILE` reads `loopmath.harness-runs/0.1` rows; an arm is a harness digest (`sha256:<hex>` or `digest:sha256:<hex>`, a prefix of one digest will do), a configuration id, or `arm=NAME` or `arm:NAME`; `check-change from-runs` takes the same forms. `--rule` takes the built-in rules or a rule defined under `[rules.NAME]` in the store's config, which the gate reads as `check-change` does, also with `--runs`. The comparison of the paired runs is `check-change`'s, and the gate adds the loop's policy to it: no adoption before `--r-min` pairs, looks at 1, 2, 4 and so on pairs up to `--r-max`, a `--floor` (the smallest gain worth adopting, 0 or more; under an acceptance rule it applies to the cut in cost per accepted result, a fraction, or to the gain in the chance of acceptance when a cost is unknown), and the dollars, `--budget-usd` for both arms (0 buys no further pair) and `--max-cost-ratio`. With no evidence, an arm that matches no run or a rule that scores no pair, it exits 1 and names the arms or the rules that are there. Both dollar flags need every run's cost and refuse, naming the runs without one; with neither flag, a run with no recorded cost leaves the decision to the outcomes alone, and the cost reads unknown, never $0. When nothing resolves within the cap or the budget, it keeps the base and says where it stopped. The next pairs' value is the one `check-change` states, valued on the way the gate goes on (its looks, `--r-min` and the 95% band), and named beside the number; `check-change from-runs` cannot express the gate's looks, so on the same runs the two numbers can differ. The value stop is on by default: from `--min-pairs` scored pairs (default 4) the gate rejects, keeping the base, when the next pairs are certified not worth their price, as `check-change --value-stop` does, and a cost that is not finite never stops it; `--no-value-stop` turns it off, and the value and its price are then shown and stop nothing. Under a score rule the value needs `--value-usd` (dollars per point of gain), as in `check-change`. In Python, `loopmath.gate.decide(base_runs, cand_runs, GatePolicy(...))` gives the same answer and writes nothing; `loopmath gate --help` lists the rest.
+- **`gate winner` (experimental).** A search loop that scores N candidates and keeps the best overstates the winner's gain: the best of N noisy scores is high partly by luck. `loopmath gate winner --scores FILE` reads every score the search made (a CSV or `loopmath.gate-winner-scores/0.1` JSON; candidates known only as no better than the best so far count too) and gives the winner's gain over its parent, shrunk by what the scores say about the candidates' spread and the noise, with 80% and 95% ranges and P(beats parent). `--runs FILE` reads harness-runs rows instead, one arm per candidate, with the spread from their replicates. Give the spread of one evaluation across reruns with `--spread X`, or an assumed range with `--spread LO:HI`. The action is `gate`'s on confirmatory runs of the parent and the winner (`--confirm FILE`), never on the search's own scores, and a projection says how likely the gate is to adopt and how many more pairs it will want. In Python, `loopmath.gate.winner.assess`; the method, the formats and the output are in [spec/gate-winner.md](spec/gate-winner.md), with worked examples from published searches in `spec/examples/gate-winner/`.
+- **DSPy's GEPA and MIPROv2 (experimental).** An optimizer's accept step keeps a new prompt when its score on a few instances goes up, and on 3 instances that keeps many children that are no better. `loopmath.integrations.dspy` puts the gate there: `GEPAAcceptance` is a GEPA acceptance criterion (run GEPA with `use_merge=False`), `MIPROv2Gate` confirms MIPROv2's replacements, and `run([], [], more)` gates any compiled program against its student. It imports neither dspy nor gepa. See [the DSPy guide](docs/dspy.md).
+- **`tasks pick` (experimental).** For a loop or a benchmark run that scores each candidate on some of a suite's tasks instead of all of them. `loopmath tasks pick --candidate SEL` reads the finished runs in the store, or `--runs FILE`, learns each task's difficulty and discrimination from the other arms' runs on it, and prints the candidate's suite score so far (the mean over the suite of its chance of acceptance per task) with 80% and 95% ranges, whether to stop, and the next task to run. From a store it writes that task's file and the `run record --task-file F --label KEY=VALUE --home PATH` line that records the run in the same store, so the next call sees it. It stops when the 80% range is as narrow as one full run of the suite would give (`--halfwidth` sets another target), or, with `--versus X` or `--versus-arm SEL`, when the score is above or below a number or another arm's at `--level`. `--method` chooses the next task: the largest expected cut in the score's variance (`loss`, the default), information, one fixed order, random, or stratified by predicted rate. The default learns fastest when the candidate behaves like the arms in the history; for a candidate unlike all of them (a new model rather than a change to a known one's harness), `--method stratified` or `random` keeps the ranges closer to honest at the cost of more sessions. A candidate that scores below every arm in the history (a broken setup, say) is estimated too high by every method, and its ranges can miss even at the full suite; under `--method random` the accepted count over the sessions, which every call prints, is the check. Every estimate prints both warnings, in text and in `--json` (`estimate.warnings`). With `--versus`, a bar within about 0.01 of the candidate's score is close to a coin toss for any method; to ask whether a change helps by at least some margin, pass the bar plus that margin. The ranges are approximate: they leave out the history fit's own error shared with other arms. `loopmath tasks pick --help` lists the rest.
+
+## Does my plugin pay?
+
+`claude plugin eval` grades a plugin's eval cases with and without the plugin. `adapt plugin-eval` (experimental) reads its result, and loopmath turns the per-case deltas into a decision: adopt, reject, or run more, with ranges on the gain and, given each run's cost with `--cost-from`, on cost per accepted result. It reads the file only and never runs Claude Code.
+
+```
+loopmath adapt plugin-eval result.json --out ocp --rows rows.jsonl
+loopmath run import ocp --finish
+loopmath check-change from-runs --base-label arm:without --cand-label arm:with
+loopmath gate --base arm:without --cand arm:with --runs rows.jsonl --rule accept --r-max 24
+```
+
+Runs of one case are not new tasks: to learn whether the plugin helps on tasks like your cases, add cases rather than runs. With grades (`--outcome score`), `check-change from-runs` compares case by case. The import's summary prints the last two filled in for your suite. Which cases are compared, what the suite's one cost can and cannot say, how far the answer reaches and the provisional input layout are in [`docs/plugin-eval.md`](docs/plugin-eval.md).
 
 ## The store and privacy
 
@@ -302,16 +330,16 @@ loopmath graph --workspace NAME --format html --out graph.html
 
 `loopmath prices` lists the packaged prices with the source of each. A rate marked "rate not confirmed" is a best public-price guess, and every dollar figure priced with one says so.
 
-`graph` rebuilds which agent launched which, and which session read what another wrote. Its formats are `ocp`, `json`, `dot`, `run` and `html`. `graph --out FILE` and `adapt NAME --out FILE` write to any path you name: missing folders are created, an existing file is replaced, and a directory is refused. `graph` prints a short summary on stderr; `--verbose` adds every count of what was skipped or could not be placed. See the [workflow graph guide](docs/graph.md) and [adapters](docs/adapters.md).
-
-## Research data
-
-`loopmath research --help` lists the verbs that read research data folders, which are not part of the package: `research fit` and `research transfer-test` read sweep run records, and `research analyze-e0` reads a session corpus. Name the folders with a flag (`--sweep-dir`, `--corpus`), an environment variable (`LOOPMATH_SWEEP_DIR`, `LOOPMATH_E0_CORPUS`), or config (`loopmath config set research.sweep_dir PATH`, `research.e0_corpus`). `research fit` and `research transfer-test` need the `bayes` extra (`pip install 'loopmath[bayes]'`), and `research analyze-e0` the `e0` extra. `loopmath prior build --out DIR` rebuilds the packaged prior bundle from research data folders like these; `loopmath prior build --help` names them.
+`graph` rebuilds which agent launched which, and which session read what another wrote. Its formats are `ocp`, `json`, `dot`, `run` and `html`. `graph --out FILE` and `adapt NAME --out FILE` write to any path you name: missing folders are created, an existing file is replaced, and a directory is refused. `graph` prints a short summary on stderr; `--verbose` adds every count of what was skipped or could not be placed. See the [workflow graph guide](docs/graph.md) and [adapters](docs/adapters.md). The ADE adapters (`adapt NAME`) are experimental: each is tested on a synthetic fixture written from its tool's format, not yet on a real store.
 
 ## More
 
 - [OCP, the run format](spec/OCP.md): the normative note that goes with the [schemas](spec/), and the [versioning promise](spec/VERSIONING.md)
+- [DSPy's GEPA and MIPROv2](docs/dspy.md): `loopmath.integrations.dspy`, the gate at their accept step
+- [Plugin eval results](docs/plugin-eval.md): `adapt plugin-eval`
+- [OpenEvolve and ShinkaEvolve runs](docs/adapters.md#openevolve-and-shinkaevolve): `adapt openevolve`, `adapt shinkaevolve` and `recommend --role`, which model to give an evolution loop's mutator
 - [Release policy](docs/release.md)
+- [Screening harness edits](docs/screen.md): `screen`, with a worked 48-session example and its limits
 - [Packaged model prices](src/loopmath/prices.toml)
 
 ## License

{loopmath-0.3.0 → loopmath-0.3.2}/README.md

@@ -136,14 +136,19 @@ The workflow builder is the one page that is served: `loopmath builder --type bu
 | Group | Commands |
 |---|---|
 | Plan | `task-types`, `workflows` (list, show, validate, diff), `recommend`, `builder` |
-| Record | `run` (start, record, import, attempt, artifact, finish, sessions), `outcome`, `budget`, `config` (get, set) |
+| Record | `run` (start, record, import, attempt, artifact, finish, sessions), `outcome`, `budget`, `config` (get, set, unset) |
 | Learn | `fit`, `onboard`, `share`, `prior` |
 | View | `runs`, `posterior`, `status`, `report`, `doctor`, `verify-receipts` |
+| Harness testing | `check-change`, `gate` (experimental), `screen` (experimental) |
+| Task subsets | `tasks pick` (experimental) |
+| Evolution loops | `adapt openevolve` (experimental), `adapt shinkaevolve` (experimental), `recommend --role` (experimental) |
 | Skill | `skill` (install, uninstall, show) |
 | OCP | `ocp` (validate, migrate) |
-| Log analysis | `analyze`, `graph`, `adapt`, `prices`, `validate-prices`, `scan` |
+| Log analysis | `analyze`, `graph`, `adapt`, `prices`, `validate-prices` |
 
-Each command prints a short summary by default. With `--json`, it prints exactly one JSON object instead. One exception: `loopmath adapt harness2` and `adapt harness-runs` without `--out` print the run documents on stdout, one per line, so their `--json` summary goes to stderr. Exit codes: 0 ok, 1 user error, 2 not found, 3 not implemented, 4 store locked, 5 no fit yet. For flags, `loopmath <command> --help` is the source of truth. Commands, pages and the `runs` table show times in your local time with the zone; the store and `--json` keep UTC. A cost under a cent shows one significant digit (`$0.004`), and an exact zero is `$0`.
+These are experimental: `gate`, `screen`, `tasks pick`, the harness-part estimates (`recommend --harness-variants` and `--harness`, `posterior --level harness` and its `factors`), `recommend --role`, `adapt harness2`, `adapt plugin-eval`, `adapt openevolve`, `adapt shinkaevolve` and the ADE adapters (`adapt NAME`). Each works and has tests, but has had little use on real work so far, and its output may change. Their help says so too. The gate at DSPy's GEPA and MIPROv2 accept step (`loopmath.integrations.dspy`, Python) is experimental too: it is tested against fakes written from their source, not yet with DSPy installed, and its guide says so. So is the opt-in cost model for arms that mix models, `loopmath.compare.model.COST_MODEL = "sw"`, a Python setting with no help of its own; the [changelog](CHANGELOG.md) describes it.
+
+Each command prints a short summary by default. With `--json`, it prints exactly one JSON object instead. One exception: `loopmath adapt harness2`, `adapt harness-runs`, `adapt plugin-eval`, `adapt openevolve` and `adapt shinkaevolve` without `--out` print the run documents on stdout, one per line, so their `--json` summary goes to stderr. Exit codes: 0 ok, 1 user error, 2 not found, 3 not implemented, 4 store locked, 5 no fit yet. For flags, `loopmath <command> --help` is the source of truth. Commands, pages and the `runs` table show times in your local time with the zone; the store and `--json` keep UTC. A cost under a cent shows one significant digit (`$0.004`), and an exact zero is `$0`.
 
 A workflow is a TOML file. `loopmath workflows list` shows the catalog and yours, which live in `~/.loopmath/workflows/`. It also lists the shapes loopmath composes from catalog pieces, such as `team`; `loopmath workflows show team` says what one is and that `recommend` does not offer it. This one implements, then sends a rejected change back once or twice:
 
@@ -217,21 +222,44 @@ A run can record the harness it ran with: the prompt files, skills, plugins, MCP
   `cost` is the multiplier on an attempt's dollars when the part is present; `success` is percentage points.
 - `loopmath recommend --harness-variants` prices the trees your runs used most, and `--harness PATH` prices one more, against the bare harness label on the same draws.
 
-An estimate is the change when a named part is present, pooled over its versions. It is not the effect of one edit, and a part on every run cannot be told apart from its harness. The cost ranges met their stated coverage in simulation. **The success ranges did not:** they pull effects toward 0, and their 80% ranges held 60% of nonzero effects. So they are marked `calibrated: false`, and so are the success-side numbers of a priced variant. With no record anywhere, every number is what it was before. The record, the digest, the factor map and the limits are in [`spec/harness.md`](spec/harness.md).
+An estimate is the change when a named part is present, pooled over its versions. It is not the effect of one edit, and a part on every run cannot be told apart from its harness. The cost ranges met their stated coverage in simulation. **The success ranges did not:** they pull effects toward 0, and their 80% ranges held 60% of nonzero effects. So they are marked `calibrated: false`, and so are the success-side numbers of a priced variant, and the harness-part estimates are experimental. With no record anywhere, every number is what it was before. The record, the digest, the factor map and the limits are in [`spec/harness.md`](spec/harness.md).
 
 ## Testing a harness change
 
 These commands are for changing the harness your agents run with and finding out whether the change paid off.
 
 - **Harness parts and `--harness`.** Record the tree each run used (`run finish --harness auto` or `PATH`), and the fit learns each named part's effect, as in the section above. `recommend --harness-variants` prices the trees your runs used on the configurations those runs used, and on the recommendation's own when their harness has data on the parts; `--harness PATH` prices one more tree. Where a tree is not priced, `not_priced` in `--json` says why. The record, the factor map and the limits are in [`spec/harness.md`](spec/harness.md).
-- **`adapt harness-runs` and `adapt harness2`.** These bring in runs another tool made. `loopmath adapt harness-runs FILE --out DIR` reads a `loopmath.harness-runs/0.1` file, one JSON line per rollout, as an auto-research loop, a plugin eval suite or a hand A/B writes it. `loopmath adapt harness2 RUNS --out DIR` reads a harness2 runs folder. Then `loopmath run import DIR --finish` stores the runs. The row format, outcomes, costs and worked examples are in [`spec/harness-runs.md`](spec/harness-runs.md).
+- **`adapt harness-runs` and `adapt harness2`.** These bring in runs another tool made; `adapt harness2` is experimental, since its input format is read from harness2's code and it has not yet read a real runs folder. `loopmath adapt harness-runs FILE --out DIR` reads a `loopmath.harness-runs/0.1` file, one JSON line per rollout, as an auto-research loop, a plugin eval suite or a hand A/B writes it. `loopmath adapt harness2 RUNS --out DIR` reads a harness2 runs folder. Then `loopmath run import DIR --finish` stores the runs. The row format, outcomes, costs and worked examples are in [`spec/harness-runs.md`](spec/harness-runs.md).
 - **`check-change`.** It decides whether a change beats what it replaces, on paired runs:
-  - `check-change start --base LABEL|PATH --cand LABEL|PATH --n N` plans the pairs and each pair's arm order;
-  - `check-change status` gives the comparison, the verdict once there is one (adopt or reject) and the next pair worth running;
+  - `check-change start --base LABEL|PATH --cand LABEL|PATH --n N` plans the pairs and each pair's arm order; `--base-set PIECE=HARNESS:MODEL:EFFORT` and `--cand-set` give each arm a setting, as `run record --set` takes it, for a model switch on one harness: the arm's runs take its set, a run with another is refused, and `status` names the models each arm ran and any run whose log ran another;
+  - `check-change status` gives the comparison, the verdict once there is one (adopt or reject) and the next pair worth running. A check that reaches its cap or spends its budget before the evidence decides keeps the base and says `not decided`. The next pair's value and price are shown at each look and stop nothing; `--value-stop` also stops a check, not decided, when the next pairs are certified not worth their price (every plan's net below 0 by more than two standard errors, or no look left can change the action), from 4 scored pairs on (`--min-pairs K`). Use it when a pair is dear against what an adoption is worth over `--horizon-runs`: in simulated checks at the default 50 future runs it saved 6 to 16 pairs a check but dropped 26 to 40 percent of the adoptions of real cuts, and at 500 it dropped none (`check-change start --help`); `--pair-price 0` compares the value with 0 for runs on a subscription, and the budget still counts the runs' recorded dollars; `--json` keeps `verdict: reject` there, with `comparison.stop` and `comparison.decided: false`;
   - `check-change from-runs --base-label SEL --cand-label SEL` compares finished runs already in the store, picked by a run label, a configuration id, a harness digest (a prefix of one digest will do) or `arm:NAME`.
 
-  `--rule` names what counts as accepted, and `loopmath check-change --help` lists the rest. A run with no dollar cost leaves the decision to the outcomes alone, and the cost reads unknown; a budget or `--objective cpar` then refuses, naming the runs without one.
-- **`gate`.** For a loop that adopts changes on its own, it answers: adopt the candidate, reject it and keep the base, or replicate N more pairs first. `loopmath gate --base SEL --cand SEL` reads the finished runs in the store, or `--runs FILE` reads `loopmath.harness-runs/0.1` rows; an arm is a harness digest (`sha256:<hex>` or `digest:sha256:<hex>`, a prefix of one digest will do), a configuration id, or `arm=NAME` or `arm:NAME`; `check-change from-runs` takes the same forms. `--rule` takes the built-in rules or a rule defined under `[rules.NAME]` in the store's config, which the gate reads as `check-change` does, also with `--runs`. The comparison of the paired runs is `check-change`'s, and the gate adds the loop's policy to it: no adoption before `--r-min` pairs, looks at 1, 2, 4 and so on pairs up to `--r-max`, a `--floor` (the smallest gain worth adopting, 0 or more; under an acceptance rule it applies to the cut in cost per accepted result, a fraction, or to the gain in the chance of acceptance when a cost is unknown), and the dollars, `--budget-usd` for both arms (0 buys no further pair) and `--max-cost-ratio`. With no evidence, an arm that matches no run or a rule that scores no pair, it exits 1 and names the arms or the rules that are there. Both dollar flags need every run's cost and refuse, naming the runs without one; with neither flag, a run with no recorded cost leaves the decision to the outcomes alone, and the cost reads unknown, never $0. When nothing resolves within the cap or the budget, it keeps the base and says where it stopped. In Python, `loopmath.gate.decide(base_runs, cand_runs, GatePolicy(...))` gives the same answer and writes nothing; `loopmath gate --help` lists the rest.
+  `--rule` names what counts as accepted, and `loopmath check-change --help` lists the rest. A run with no dollar cost leaves the decision to the outcomes alone, and the cost reads unknown; a budget or `--objective cpar` then refuses, naming the runs without one. When an arm's runs mix a cheap model and a dear one, the default cost model can put the dearer arm's cost per run below the cheaper arm's. In Python, the experimental setting `loopmath.compare.model.COST_MODEL = "sw"` reads such costs for `compare.paired` and `gate.decide`; the [changelog](CHANGELOG.md) gives its limits.
+- **`screen` (experimental).** It tests many harness edits at once. Put each edit in its own subfolder of an edits folder, laid over a base harness tree: `PATH` copies a file, `PATH.append` appends to one, `PATH.remove` removes lines. Then:
+  - `screen start --base DIR --edits DIR --tasks FILE.jsonl --rule tests` plans one fold-over pair per task: arm A runs the edits its row turns on, arm B the others, so task difficulty cancels in the pair and every session informs every edit. 12, 16, 20, 24 or 32 pairs for up to one edit fewer give orthogonal rows;
+  - `screen next` prints the next pair, with both arms' trees built in the store; `run record --screen SCR --arm A|B` records a run into it;
+  - `screen status` ranks the edits on success and cost with 80% ranges and P(helps), names the predicted-best subset with its predicted cut in cost per accepted result (often a subset no arm ran), and prices the next pair. `--on EDIT,EDIT` predicts any subset;
+  - `screen apply --best --out DIR` writes the base with the best edits, ready for a `check-change` against the base.
+
+  The model is a Bayesian linear model of each pair's difference, on success and on log cost, with its noise and effect scales weighed over a grid rather than plugged in. A finished orthogonal screen weighs two views, a few edits matter (sparse) or all matter a little (dense), by how well each explains the pairs, and `best.by_view` shows both. On a dozen pairs each edit's effect is shrunk toward 0 (a large one by about half with 8 edits in 12 pairs), but in simulations at that size, of 1,000 screens drawn from effect settings no tuning used, 964 picked a best subset and its 80% range held the true cut in 812 (84%), a few points high where the true cut was small and low where it was large; the other 36 picked no edit, which predicts no change. The prediction is against the base, which no arm runs, so an effect of having the edited files at all is not in it. The [screen guide](docs/screen.md) has a worked example and the limits.
+- **`gate` (experimental).** For a loop that adopts changes on its own, it answers: adopt the candidate, reject it and keep the base, or replicate N more pairs first. `loopmath gate --base SEL --cand SEL` reads the finished runs in the store, or `--runs FILE` reads `loopmath.harness-runs/0.1` rows; an arm is a harness digest (`sha256:<hex>` or `digest:sha256:<hex>`, a prefix of one digest will do), a configuration id, or `arm=NAME` or `arm:NAME`; `check-change from-runs` takes the same forms. `--rule` takes the built-in rules or a rule defined under `[rules.NAME]` in the store's config, which the gate reads as `check-change` does, also with `--runs`. The comparison of the paired runs is `check-change`'s, and the gate adds the loop's policy to it: no adoption before `--r-min` pairs, looks at 1, 2, 4 and so on pairs up to `--r-max`, a `--floor` (the smallest gain worth adopting, 0 or more; under an acceptance rule it applies to the cut in cost per accepted result, a fraction, or to the gain in the chance of acceptance when a cost is unknown), and the dollars, `--budget-usd` for both arms (0 buys no further pair) and `--max-cost-ratio`. With no evidence, an arm that matches no run or a rule that scores no pair, it exits 1 and names the arms or the rules that are there. Both dollar flags need every run's cost and refuse, naming the runs without one; with neither flag, a run with no recorded cost leaves the decision to the outcomes alone, and the cost reads unknown, never $0. When nothing resolves within the cap or the budget, it keeps the base and says where it stopped. The next pairs' value is the one `check-change` states, valued on the way the gate goes on (its looks, `--r-min` and the 95% band), and named beside the number; `check-change from-runs` cannot express the gate's looks, so on the same runs the two numbers can differ. The value stop is on by default: from `--min-pairs` scored pairs (default 4) the gate rejects, keeping the base, when the next pairs are certified not worth their price, as `check-change --value-stop` does, and a cost that is not finite never stops it; `--no-value-stop` turns it off, and the value and its price are then shown and stop nothing. Under a score rule the value needs `--value-usd` (dollars per point of gain), as in `check-change`. In Python, `loopmath.gate.decide(base_runs, cand_runs, GatePolicy(...))` gives the same answer and writes nothing; `loopmath gate --help` lists the rest.
+- **`gate winner` (experimental).** A search loop that scores N candidates and keeps the best overstates the winner's gain: the best of N noisy scores is high partly by luck. `loopmath gate winner --scores FILE` reads every score the search made (a CSV or `loopmath.gate-winner-scores/0.1` JSON; candidates known only as no better than the best so far count too) and gives the winner's gain over its parent, shrunk by what the scores say about the candidates' spread and the noise, with 80% and 95% ranges and P(beats parent). `--runs FILE` reads harness-runs rows instead, one arm per candidate, with the spread from their replicates. Give the spread of one evaluation across reruns with `--spread X`, or an assumed range with `--spread LO:HI`. The action is `gate`'s on confirmatory runs of the parent and the winner (`--confirm FILE`), never on the search's own scores, and a projection says how likely the gate is to adopt and how many more pairs it will want. In Python, `loopmath.gate.winner.assess`; the method, the formats and the output are in [spec/gate-winner.md](spec/gate-winner.md), with worked examples from published searches in `spec/examples/gate-winner/`.
+- **DSPy's GEPA and MIPROv2 (experimental).** An optimizer's accept step keeps a new prompt when its score on a few instances goes up, and on 3 instances that keeps many children that are no better. `loopmath.integrations.dspy` puts the gate there: `GEPAAcceptance` is a GEPA acceptance criterion (run GEPA with `use_merge=False`), `MIPROv2Gate` confirms MIPROv2's replacements, and `run([], [], more)` gates any compiled program against its student. It imports neither dspy nor gepa. See [the DSPy guide](docs/dspy.md).
+- **`tasks pick` (experimental).** For a loop or a benchmark run that scores each candidate on some of a suite's tasks instead of all of them. `loopmath tasks pick --candidate SEL` reads the finished runs in the store, or `--runs FILE`, learns each task's difficulty and discrimination from the other arms' runs on it, and prints the candidate's suite score so far (the mean over the suite of its chance of acceptance per task) with 80% and 95% ranges, whether to stop, and the next task to run. From a store it writes that task's file and the `run record --task-file F --label KEY=VALUE --home PATH` line that records the run in the same store, so the next call sees it. It stops when the 80% range is as narrow as one full run of the suite would give (`--halfwidth` sets another target), or, with `--versus X` or `--versus-arm SEL`, when the score is above or below a number or another arm's at `--level`. `--method` chooses the next task: the largest expected cut in the score's variance (`loss`, the default), information, one fixed order, random, or stratified by predicted rate. The default learns fastest when the candidate behaves like the arms in the history; for a candidate unlike all of them (a new model rather than a change to a known one's harness), `--method stratified` or `random` keeps the ranges closer to honest at the cost of more sessions. A candidate that scores below every arm in the history (a broken setup, say) is estimated too high by every method, and its ranges can miss even at the full suite; under `--method random` the accepted count over the sessions, which every call prints, is the check. Every estimate prints both warnings, in text and in `--json` (`estimate.warnings`). With `--versus`, a bar within about 0.01 of the candidate's score is close to a coin toss for any method; to ask whether a change helps by at least some margin, pass the bar plus that margin. The ranges are approximate: they leave out the history fit's own error shared with other arms. `loopmath tasks pick --help` lists the rest.
+
+## Does my plugin pay?
+
+`claude plugin eval` grades a plugin's eval cases with and without the plugin. `adapt plugin-eval` (experimental) reads its result, and loopmath turns the per-case deltas into a decision: adopt, reject, or run more, with ranges on the gain and, given each run's cost with `--cost-from`, on cost per accepted result. It reads the file only and never runs Claude Code.
+
+```
+loopmath adapt plugin-eval result.json --out ocp --rows rows.jsonl
+loopmath run import ocp --finish
+loopmath check-change from-runs --base-label arm:without --cand-label arm:with
+loopmath gate --base arm:without --cand arm:with --runs rows.jsonl --rule accept --r-max 24
+```
+
+Runs of one case are not new tasks: to learn whether the plugin helps on tasks like your cases, add cases rather than runs. With grades (`--outcome score`), `check-change from-runs` compares case by case. The import's summary prints the last two filled in for your suite. Which cases are compared, what the suite's one cost can and cannot say, how far the answer reaches and the provisional input layout are in [`docs/plugin-eval.md`](docs/plugin-eval.md).
 
 ## The store and privacy
 
@@ -264,16 +292,16 @@ loopmath graph --workspace NAME --format html --out graph.html
 
 `loopmath prices` lists the packaged prices with the source of each. A rate marked "rate not confirmed" is a best public-price guess, and every dollar figure priced with one says so.
 
-`graph` rebuilds which agent launched which, and which session read what another wrote. Its formats are `ocp`, `json`, `dot`, `run` and `html`. `graph --out FILE` and `adapt NAME --out FILE` write to any path you name: missing folders are created, an existing file is replaced, and a directory is refused. `graph` prints a short summary on stderr; `--verbose` adds every count of what was skipped or could not be placed. See the [workflow graph guide](docs/graph.md) and [adapters](docs/adapters.md).
-
-## Research data
-
-`loopmath research --help` lists the verbs that read research data folders, which are not part of the package: `research fit` and `research transfer-test` read sweep run records, and `research analyze-e0` reads a session corpus. Name the folders with a flag (`--sweep-dir`, `--corpus`), an environment variable (`LOOPMATH_SWEEP_DIR`, `LOOPMATH_E0_CORPUS`), or config (`loopmath config set research.sweep_dir PATH`, `research.e0_corpus`). `research fit` and `research transfer-test` need the `bayes` extra (`pip install 'loopmath[bayes]'`), and `research analyze-e0` the `e0` extra. `loopmath prior build --out DIR` rebuilds the packaged prior bundle from research data folders like these; `loopmath prior build --help` names them.
+`graph` rebuilds which agent launched which, and which session read what another wrote. Its formats are `ocp`, `json`, `dot`, `run` and `html`. `graph --out FILE` and `adapt NAME --out FILE` write to any path you name: missing folders are created, an existing file is replaced, and a directory is refused. `graph` prints a short summary on stderr; `--verbose` adds every count of what was skipped or could not be placed. See the [workflow graph guide](docs/graph.md) and [adapters](docs/adapters.md). The ADE adapters (`adapt NAME`) are experimental: each is tested on a synthetic fixture written from its tool's format, not yet on a real store.
 
 ## More
 
 - [OCP, the run format](spec/OCP.md): the normative note that goes with the [schemas](spec/), and the [versioning promise](spec/VERSIONING.md)
+- [DSPy's GEPA and MIPROv2](docs/dspy.md): `loopmath.integrations.dspy`, the gate at their accept step
+- [Plugin eval results](docs/plugin-eval.md): `adapt plugin-eval`
+- [OpenEvolve and ShinkaEvolve runs](docs/adapters.md#openevolve-and-shinkaevolve): `adapt openevolve`, `adapt shinkaevolve` and `recommend --role`, which model to give an evolution loop's mutator
 - [Release policy](docs/release.md)
+- [Screening harness edits](docs/screen.md): `screen`, with a worked 48-session example and its limits
 - [Packaged model prices](src/loopmath/prices.toml)
 
 ## License

{loopmath-0.3.0 → loopmath-0.3.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "loopmath"
-version = "0.3.0"
+version = "0.3.2"
 description = "Plan agent workflows, record how they went, and learn from your own Claude Code and Codex logs"
 readme = "README.md"
 license = "MIT"

{loopmath-0.3.0 → loopmath-0.3.2}/src/loopmath/__init__.py

@@ -5,4 +5,4 @@ The command line (`loopmath`, or its alias `loop`) is the interface; see the
 README and `loopmath --help`.
 """
 
-__version__ = "0.3.0"
+__version__ = "0.3.2"

loopmath-0.3.2/src/loopmath/adapters/evolve.py

@@ -0,0 +1,137 @@
+"""What the OpenEvolve and ShinkaEvolve readers share: one harness-runs row per LLM call whose model was recorded
+(an OpenEvolve child, a ShinkaEvolve patch query), with one `mutator` stage, and the outcome "this call wrote a child
+that beat its parent".
+
+The rows use the run log's `stages` (spec/harness-runs.md): the role is `mutator`, the piece `mutate`, the agent
+the tool's name and the model the one that wrote the child. A verdict outcome says whether the child's score beat
+its parent's by more than `min_delta`; `delta`, `parent_score` and `child_score` ride along as numbers. Nothing
+the tool wrote as text (code, diffs, prompts, replies, feedback, logs) is copied: ext holds ids, numbers and
+loopmath's own labels.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import math
+import re
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from typing import Any, Callable
+
+ROLE = "mutator"
+PIECE = "mutate"
+EXPERIMENT_LABEL = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$")
+
+
+class FormatError(ValueError):
+    """An input the reader does not recognise, named with what it found."""
+
+
+class NeedsFlag(ValueError):
+    """The input is readable, but a flag must say what it does not: the message names the flag."""
+
+
+@dataclass(frozen=True)
+class TablePrice:
+    """A price table the caller loaded, as a reader uses it: `usd(model, tokens)` gives the dollars for one
+    model's tokens (keys `in`, `cache_read`, `cache_write` and `out`), or None when the table has no row for the
+    model; `path` and `as_of` name the table in the notes. The readers never load or price a table themselves."""
+
+    usd: Callable[[str, dict[str, int]], float | None]
+    path: str
+    as_of: str
+
+
+@dataclass
+class Result:
+    rows: list[dict[str, Any]]
+    format: str
+    skipped: list[dict[str, str]] = field(default_factory=list)   # [{run_id, reason}]
+    counts: dict[str, int] = field(default_factory=dict)
+    experiments: list[dict[str, Any]] = field(default_factory=list)
+    notes: list[str] = field(default_factory=list)
+    accounting: dict[str, float] = field(default_factory=dict)   # dollars: total, attributed, unattributed
+
+
+def number(value: Any) -> float | None:
+    """A finite float, or None for anything else (bools are flags, not scores)."""
+    if isinstance(value, bool) or not isinstance(value, (int, float)):
+        return None
+    value = float(value)
+    return value if math.isfinite(value) else None
+
+
+def iso(epoch: Any) -> str | None:
+    """Epoch seconds as RFC 3339 UTC, or None."""
+    value = number(epoch)
+    if value is None or value <= 0:
+        return None
+    return datetime.fromtimestamp(value, tz=timezone.utc).isoformat().replace("+00:00", "Z")
+
+
+def experiment_id(prefix: str, root_id: str) -> str:
+    """A stable experiment id from the run's first program: `<prefix>-` and 16 hex."""
+    return f"{prefix}-" + hashlib.sha256(root_id.encode("utf-8")).hexdigest()[:16]
+
+
+def check_label(label: str | None) -> None:
+    if label is not None and not EXPERIMENT_LABEL.match(label):
+        raise NeedsFlag(f"--experiment {label!r}: letters, digits, '.', '_' or '-', at most 64")
+
+
+def improved(child: float | None, parent: float | None, min_delta: float) -> tuple[bool, float | None]:
+    """Whether the child beat its parent by more than `min_delta`, and the delta (None when either is unknown)."""
+    if child is None or parent is None:
+        return False, None
+    delta = child - parent
+    return delta > min_delta, delta
+
+
+def mutator_row(*, tool: str, ns: str, experiment: str, child_id: str, task_id: str, task: dict[str, Any],
+                model: str, effort: str | None, accepted: bool, delta: float | None, parent_score: float | None,
+                child_score: float | None, metric: str | None, cost: dict[str, Any] | None, status: str,
+                started: str | None, ended: str | None, stage_started: str | None, stage_ended: str | None,
+                stage: str, note: dict[str, Any]) -> dict[str, Any]:
+    """One harness-runs row (spec/harness-runs.md) for one child: a single mutator stage."""
+    from ..harness_runs.rows import SCHEMA
+
+    outcome: dict[str, Any] = {"accepted": accepted, "by": tool}
+    if metric:
+        outcome["metric"] = metric
+    for key, value in (("delta", delta), ("parent_score", parent_score), ("child_score", child_score)):
+        if value is not None:
+            outcome[key] = value
+    st: dict[str, Any] = {"role": ROLE, "piece": PIECE, "status": status}
+    if cost:
+        st["cost"] = cost
+    if stage_started:
+        st["started_at"] = stage_started
+    if stage_ended:
+        st["ended_at"] = stage_ended
+    row: dict[str, Any] = {"schema": SCHEMA, "id": f"{tool}:{experiment}:{child_id}", "task_id": task_id,
+                           "task": task, "experiment": experiment, "status": status, "agent": tool, "model": model,
+                           "outcome": outcome, "stages": [st],
+                           "tool": {"name": tool, "run_id": child_id, "task_id": task_id, "stage": stage},
+                           "ext": {ns: {k: v for k, v in note.items() if v is not None}}}
+    if effort:
+        row["effort"] = effort
+    if started:
+        row["started_at"] = started
+    if ended:
+        row["ended_at"] = ended
+    return row
+
+
+def task_of(task_id: str, task_type: str | None, repo: str | None) -> dict[str, Any]:
+    return {"type": task_type or "research", "repo": repo or task_id}
+
+
+def experiments(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    counts: dict[str, int] = {}
+    for r in rows:
+        counts[r["experiment"]] = counts.get(r["experiment"], 0) + 1
+    return [{"experiment": e, "runs": n} for e, n in sorted(counts.items())]
+
+
+def plural(n: int, word: str, many: str | None = None) -> str:
+    return f"{n} {word if n == 1 else (many or word + 's')}"